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FOREWORD 


This  study  applies  judgment  and  decision  making  research 
techniques  to  military  officers'  understanding  of  factors  in  the 
battlefield  pertinent  to  the  success  of  planned  courses  of 
action.  The  need  for  judgments  of  probability  and  of  relative 
importance  is  widespread  in  operational  contexts.  The  work  was 
carried  out  in  the  Fort  Leavenworth  Field  Unit  of  the  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  Systems 
Research  Laboratory,  whose  mission  is  to  improve  the  efficiency, 
accuracy,  and  timeliness  of  command  and  control  (C^)  .  The  work 
was  performed  under  the  auspices  of  the  National  Research  Council 
Associateship  program. 
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JUDGMENTS  OF  PROBABILITY  AND  RELATIVE  IMPORTANCE  IN  A  MILITARY 
DECISION  SCENARIO:  THE  INFLUENCE  OF  SUBJECTIVE  AND  OBJECTIVE 
VARIATIONS  IN  CAUSAL  FACTORS 


EXECUTIVE  SUMMARY 


Requirement: 

A  key  function  in  command  and  control  is  monitoring  the 
uncertain  and  changing  battlefield  so  that  one  may  act 
appropriately.  Communication  about  the  battlefield  situation 
uses  expressions  of  probability  to  discuss  uncertainty,  and 
expressions  of  relative  importance  to  direct  attention  to  factors 
that  potentially  play  key  roles  in  determining  the  outcome  of 
battle.  In  order  to  be  meaningful,  these  communications  must 
have  certain  characteristics.  For  probability,  these  include 
accuracy,  coherence,  and  independence  from  irrelevant  influences. 
For  relative  importance  judgments,  a  desirable  characteristic  is 
that  judgments  be  global  or  general,  rather  than  local  or 
particular.  A  study  was  done  to  test  these  features  of 
communication  about  probability  and  relative  importance. 


Procedure : 

Subjects  were  294  students  of  the  Command  and  General  Staff 
College,  and  for  comparison  154  undergraduate  college  students. 
Each  read  a  description  in  which  a  commander  had  to  respond  to  a 
situation.  Two  problems  were  used,  one  about  a  response  to  a 
crashed  helicopter,  the  other  about  an  attack  across  a  river. 

For  each  problem,  the  subject  estimated  the  mission's  probability 
of  success  and  the  relative  importance  of  eight  situational 
factors . 

The  situation  descriptions  were  varied  between  subjects  to 
manipulate  the  probability  of  mission  success.  In  one  set  of 
conditions  the  strength  of  the  enemy  was  varied.  For  example,  in 
the  river  problem  the  enemy  was  said  to  be  at  50%,  70%,  or  90% 
strength.  In  another  set  of  conditions,  the  mood  of  the 
presentation  was  varied  by  changing  the  description  of  the 
situation  and  the  characters'  feelings  and  remarks  without 
changing  the  facts  of  the  situation. 


Vll 


Findings: 


Subjects'  judgments  of  probability  of  mission  success  were 
not  sensitive  to  the  enemy  strength.  However,  on  one  of  the 
problems  they  were  sensitive  to  variations  in  the  subjective 
factors.  While  the  most  direct  evidence  indicated  that  subjects 
make  global  relative  importance  judgments,  indirect  evidence  (a 
"halo  effect")  showed  local  effects,  that  is,  that  the  relative 
importance  of  factors  was  influenced  by  the  particular  situation 
rather  than  being  constant  across  variations  in  the  situation. 
College  students'  judgments  were  similar  to  military  officers' 
except  that  the  students  reported  greater  probabilities  of 
success . 


Utilization  of  Findings: 

The  results  suggest  that  there  is  a  need  to  train  Army 
officers  to  use  probabilities  better  in  operational  contexts. 

For  the  most  part,  relative  importance  measures  are  stable  across 
changes  in  the  situation,  indicating  that  they  may  be  useful  for 
general  analysis  of  command  and  control.  In  situations  where 
local  judgments  of  relative  importance  are  appropriate,  it  may  be 
better  to  communicate  in  terms  of  changes  in  probability  than  to 
use  a  general  vocabulary  of  importance. 
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JUDGMENTS  OF  PROBABILITY  AND  RELATIVE  IMPORTANCE  IN  A 
MILITARY  DECISION  SCENARIO:  THE  INFLUENCE  OF 
SUBJECTIVE  AND  OBJECTIVE  VARIATIONS  IN  CAUSAL 

FACTORS 

Introduction 

Monitoring  of  uncertain  and  changing  situations  is  a  key  function 
of  command  and  control  (Crumley,  1989;  Olmstead,  Christensen,  and 
Lackey,  1973) .  Because  much  battlefield  knowledge  can.  hot  be 
certain,  a  way  of  gauging  uncertainty  is  required.  And  because 
the  point  of  command  is  to  direct  action  toward  a  goal, 
estimation  of  the  relative  importance  of  different  factors  in  the 
situation  that  influence  the  attainment  of  the  goal  is  also 
needed.  "An  active  leader  will  keep  abreast  of  uhe  situation  at 
all  times  and  weight  pertinent  factors  at  any  given  time  when 
action  on  his  part  is  required"  (Major  General  John  B.  Coulder, 
1948;  quoted  by  Campbell  and  McKinney,  1990,  p  57).  Reliable 
communication  of  these  assessments  of  uncertainty  and  importance 
is  necessary. 

Uncertainty  or  degree  of  belief  may  be  measured  using  expressions 
of  probability.  A  numerical  measure  of  probability  has  strong 
advantages  over  alternative  measures  (Hogarth,  1987) .  But 
people’s  subjective  numerical  probabilities  are  not  always 
accurate.  Poor  calibration  has  frequently  been  demonstrated  when 
people  assess  their  iegree  of  belief  in  the  accuracy  of  their 
knowledge  of  simple  facts  (Lichtenstein,  Fischhoff,  and  Phillips, 
1982) ,  but  also  with  judgments  concerning  complex  situations 
about  which  the  judge  has  extensive  knowledge  (Poses,  Bekes, 
Copare,  and  Scott,  1990;  Tape  and  Wigton,  1989).  It  has  been 
demonstrated,  however,  that  those  with  much  experience  assessing 
probabilities  of  weather  (Murphy  and  Winkler,  1977)  or  patient 
mortality  in  an  intensive  care  unit  (McClish  and  Powell,  1989) 
are  quite  accurate.  Perhaps  unconfident  in  their  numerical 
expressions  of  probability,  people  often  prefer  verbal 
expressions  (Wallsten,  Budescu,  Rapoport,  Zvick,  and  Forsyth, 
1986)  or  avoid  measuring  it  at  all  (Kuipers,  Moskowitz,  and 
Kassirer,  1988) .  Given  the  potential  utility  of  subjective 
probability  assessments  in  military  operations,  and  the  uneven 
quality  of  people's  probability  judgments  in  other  contexts,  it 
is  of  interest  to  know  how  well  military  officers  can  judge  the 
numerical  probabilities  of  complex  battlefield  events. 

Judgments  of  relative  importance.  The  definition  of  a  measure  of 
relative  importance  depends  on  the  context  in  which  it  is  used 
(Hamm,  1990c) .  The  prediction  context  is  especially  pertinent  to 
command  and  control;  in  the  causal  structure  of  a  battlefield, 
some  factors  are  more  important  in  controlling  the  outcome  of  the 
battle  than  others.  One  person's  judgments  of  relative  importance 
can  guide  others  in  focusing  attention  or  effort  when  managing 
(or  coping  with)  the  battle.  A  small  number  of  researchers  ha^'e 
investigated  relative  importance  judgments  in  the  prediction 
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context  (Fischhoff,  1975;  Hamm,  1990a,  1990d;  Wasserman,  Lempert, 
and  Hastie,  1990) .  In  related  work  involving  judgments  of  the 
importance  of  story  elements  (sentences  or  phrases)  for  the 
meaning  of  the  story  (Trabasso  and  Sperry,  1985) ,  and  of  the 
importance  of  evidence  elements  for  a  decision  about  guilt  or 
innocence  (Pennington  and  Hastie,  1988),  the  element's  importance 
has  proven  to  be  related  to  its  causal  role  in  the  story.  This 
suggests  that  people  will  find  it  natural  to  judge  relative 
importance  in  a  predictive  context. 

Much  past  work  on  subjective  judgments  of  relative  importance  has 
focussed  on  the  preference  context  (e.g.,  what  factors  in  a 
battle  outcome  are  more  important  in  determining  how  good  the 
outcome  is;  see  for  example  Anderson  and  Zalinski,  1988; 
Goldstein,  in  press;  Reilly  and  Doherty,  1989;  Schmitt  and 
Levine,  1977) .  Issues  raised  in  the  preference  context  are 
pertinent  to  the  judgment  of  relative  importance  in  the 
prediction  context. 

One  issue  is  whether  people  make  local  or  global  judgments  of 
relative  importance.  In  a  global  interpretation,  relative 
importance  is  "a  fixed  attitude  of  the  decision  maker,  a  stable 
characteristic  that  doesn't  depend  on  the  particular  stimuli 
involved,  provided  that  the  stimuli  don't  disturb  the  person's 
implicit  contextual  assumptions"  (Goldstein,  in  press) .  A 
subjective  relative  importance  judgment  that  is  global,  in  that 
it  could  be  applied  to  a  new  situation  to  produce  valid  causal 
predictions,  could  be  very  useful  for  instructing  an  agent  to 
carry  out  one's  preferences  (Goldstein  and  Beattie,  in  press)  or 
to  predict  outcomes  the  same  way  that  an  expert  would  predict 
them. 

In  a  local  interpretation  of  relative  importance,  on  the  other 
hand,  a  factor's  importance  could  be  determined  by  the  particular 
characteristics  of  that  factor  in  the  situation  under 
consideration  (Goldstein,  in  press) .  For  example,  using  a  local 
definition  of  relative  importance,  fuel  supply  could  be  seen  as 
more  important  in  a  situation  where  the  fuel  tanks  were  almost 
empty  than  in  a  situation  where  the  tanks  were  full.  Generally, 
the  configuration  of  causes  in  the  situation  would  invoke  a 
particular  pattern  or  "pragmatic  reasoning  schema"  (c.f.  Cheng, 
Holyoak,  Nisbett,  and  Oliver,  1986) ,  concerning  evaluation  of 
potential  losses  and  gains.  Each  such  schema  would  provide  a 
different  basis  for  calling  aspects  of  the  situation  important. 

In  contrast,  a  global  definition  of  importance  would  call  fuel 
supply  equally  important  in  both  situations,  because  it  takes 
account  of  the  possibility  of  fuel  shortage  whether  or  not  fuel 
is  short  in  the  present  situation. 

An  alternative  perspective  on  local  versus  global  definitions  of 
relative  importance  distinguishes  them  by  asking,  "What  is  the 
implicit  population  of  situations  for  which  the  importance  rating 
is  expected  to  apply?"  In  this  light,  "local"  relative  importance 
applies  to  a  set  of  situations  whose  features  vary  only  slightly 
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from  the  present  situation,  while  the  ’'global"  relative 
importance  applies  to  a  broader  set  of  situations.  Goldstein  (in 
press)  showed  that  people’s  judgments  of  relative  importance,  in 
a  preference  context,  have  characteristics  of  a  local  rather  than 
a  global  definition.  However,  it  was  possible  to  change  this  by 
giving  instructions  emphasizing  that  the  judgments  were  going  to 
be  used  in  another  context  (Goldstein  and  Mitzel,  in  press).  In 
effect,  these  instructions  called  the  subjects'  attention  to  a 
broader  set  of  situations. 

For  some  purposes  a  global  interpretation  of  relative  importance 
is  essential.  Coates  and  McCourt  (1976)  elicited  global  relative 
importance  judgments  concerning  the  value  of  different  aspects  of 
order-of-battle  intelligence.  Dockery  and  Murray  (1987)  used 
approximate  judgments  of  the  degree  to  which  a  capability 
supports  a  particular  aspect  of  the  defense  mission  as  a  basis 
for  fuzzy  relative  importance  in  a  computerized  decision  aid  for 
assessing  the  state  of  support.  If  informants  had  provided 
local  relative  importance  judgments  for  either  of  these  projects, 
then  their  judgments  would  have  depended  on  the  situations  they 
happened  to  be  thinking  about,  and  the  projects'  analyses  would 
have  no  general  applicability. 

For  the  purposes  of  command  and  control  in  a  particular  battle,  a 
relatively  local  interpretation  of  predictive  relative  importance 
may  be  more  useful  than  a  global  one.  Local  relative  importance 
may  give  immediate  guidance  to  decisions  about  how  to  allocate 
effort  in  order  to  influence  the  outcome,  while  global  relative 
importance  may  require  interpretation  before  it  is  applicable. 
Interpretation  requires  effort  (hence  might  not  be  done)  and  is 
subject  to  error.  A  further  issue  is  whether  an  alternative 
judgment,  such  as  "impact"  (Goldstein,  in  press;  Hamm,  Bursztajn, 
Mills,  Appelbaum,  and  Gutheil,  1981)  or  more  specifically  "change 
in  probability  of  success"  (Hamm,  1990c) ,  would  be  more  useful 
for  focussing  effort  in  a  battlefield  context  than  a  judgment  of 
"relative  importance". 

Standards  for  judgments  in  complex,  unique  situations.  It  is 

essential  that  communications  about  uncertainty  and  importance  be 
both  understood  (in  that  the  recipient  understands  what  the 
sender  means)  and  accurate  (in  that  what  the  sender  says  and  what 
the  receiver  hears  correspond  to  the  situation) .  Although  in  a 
controlled  laboratory  situation  it  may  be  easy  to  measure  what  is 
more  probable  or  more  important,  with  realistic  situations  such 
standards  are  hard  to  come  by.  However,  a  rough  evaluation  of  the 
quality  of  military  officers'  subjective  assessments  of  the 
situation  can  be  provided  using  analysis  of  the  direction  in 
which  a  reasonable  judgment  would  change  in  the  face  of  a  change 
in  the  situation. 

Prudence  would  demand  that  people's  probability  judgments  be 
sensitive  to  changes  in  major  causal  aspects  of  the  situation. 

For  example,  an  officer  facing  a  small  force  should  believe  with 
a  higher  level  of  probability  that  his  mission  will  succeed  than 


an  officer  facing  a  large  force,  if  all  other  factors  in  the  two 
officers'  situations  are  equal. 

Whether  people's  importance  judgments  should  be  sensitive  to  a 
change  in  a  causal  factor  depends  on  whether  importance  is 
defined  locally  or  globally.  An  officer  attacking  an  enemy  on  the 
move  in  the  open  should  assess  that  his  or  her  attack  helicopters 
are  more  important  (in  a  local  sense)  than  those  of  an  officer 
attacking  a  well  prepared  defense.  However,  if  the  officer  is 
evaluating  the  importance  of  attack  helicopters  in  general,  it 
should  not  matter  whether  he  or  she  happens  to  be  thinking  about 
an  exposed  or  a  defended  enemy,  because  the  importance  judgment 
should  cover  both  cases. 

Further,  locally-  and  globally-defined  relative  importance 
judgments  should  each  be  independent  of  trivial  aspects  of  the 
presentation  of  the  judgment  problem.  That  is,  there  are  some 
changes  in  the  situation  that  should  not  affect  relative 
importance  judgments.  For  example,  morale  or  mood  is  an  important 
aspect  of  the  functioning  of  military  forces.  Nonetheless,  an 
officer's  understanding  of  a  situation,  including  his  or  her 
subjective  judgments  of  the  probability  of  success  and  of  the 
relative  importance  of  various  causal  factors,  should  be  based  on 
the  solid  information  that  is  available  rather  than  on  the  mood 
of  the  other  people  in  the  command  post,  unless  that  mood  plays  a 
causal  role  in  the  outcome.  The  Vincennes  commander's  decision  to 
fire  at  the  approaching  airplane  that  turned  out  to  be  a  civilian 
Iranian  airliner  was  influenced  by  the  stress  and  expectations  on 
the  bridge  (Fogarty,  1988;  Nisbett,  1988),  and  provides  an 
extreme  example  of  how  judgments  of  probability  or  relative 
importance  might  be  influenced  by  a  general  mood  unrelated  to  the 
actual  facts  of  the  situation. 

Issues  addressed  by  the  study.  The  present  study  investigated  the 
accuracy  of  Army  officers'  subjective  judgments  of  probability, 
and  whether  their  relative  importance  judgments  are  local  or 
global.  Subjects  were  presented  with  a  military  battlefield 
situation  and  asked  for  their  judgments  of  the  probability  of 
mission  success  and  of  the  importance  of  several  factors  in  the 
battlefield  situation.  In  alternative  versions  of  the  situation 
presented  to  different  subjects,  the  enemy's  threat  was  varied. 

Do  the  subjects'  assessments  of  the  probability  of  success  vary 
as  a  function  of  this  key  factor?  Does  their  judgment  of  the 
importance  of  this  factor  vary?  Another  variable  was  an 
optimistic  or  pessimistic  presentation  of  the  situation.  Where 
the  changes  in  the  objective  situation  are  expected  to  influence 
the  subjects'  probabilities,  and  possibly  their  relative 
importance  judgments,  the  changes  in  mood  should  not  have  such  an 
effect. 

The  strength  of  the  enemy,  specifically  the  enemy's  numbers  in 
comparison  with  one's  own  numbers,  has  long  been  recognized  as  an 
important  predictor  of  the  outcome  of  battle.  Methods  of 
calculating  the  force  ratio  at  particular  locations  in  the 
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battlefield  are  taught  as  part  of  the  curriculum  in  the  Command 
and  General  Staff  College,  which  the  subjects  of  this  study  were 
attending.  There  has  been  technical  debate  concerning  the 
validity  of  such  general  rules  for  estimating  the  probability  of 
success  as  a  function  of  the  ratio  of  numbers  of  the  attacking 
and  defending  forces  (Helmbold,  1969;  Stockfisch,  1975) .  The 
forces'  experience  in  realistic  training  exercises,  for  example, 
can  exercise  more  influence  than  relative  numbers  (Sulzen,  1987) . 
Nonetheless,  because  relative  strength  is  on  the  face  of  it  a 
determining  factor  of  battle  outcome,  and  because  the  subjects 
had  had  explicit  training  in  it,  it  was  used  as  the  objective 
factor  that  was  varied  in  this  study. 

If  subjective  probability  is  to  be  a  reliable  form  of 
communication  about  uncertainty,  people's  judgments  of  the 
probability  of  an  outcome  in  a  situation  should  not  vary  when  the 
same  information  is  presented  in  different  orders.  Thus,  it 
should  make  no  difference  whether  favorable  details  of  a 
situation  are  presented  first  and  unfavorable  last,  or  vice 
versa.  Research  with  military  intelligence  analysts  has  shown, 
however,  that  late  arriving  information  has  less  impact  on  a 
probability  judgment  (Tolcott,  Marvin,  and  Lehner,  1989)  .  A 
related  form  of  information  variation  is  whether  a  fact  is 
considered  well  established  or  hypothetical.  This  too  should  not 
affect  people's  probability  judgments.  For  example,  one  should 
judge  a  coin  flip  as  50%  likely  to  be  heads,  no  matter  whether 
one  is  told  "the  coin  will  definitely  be  flipped"  or  "suppose  we 
were  to  flip  the  coin".  Similarly,  one  should  assign  the  same 
probabilities  to  outcomes  in  a  situation  with  a  particular 
constellation  of  factors,  no  matter  whether  that  constellation  is 
the  best  available  description  of  reality,  or  a  hypothetical 
description  known  to  be  different  in  one  or  more  details  from  the 
actual  case.  However,  one  cognitive  mechanism  by  which  people 
make  subjective  judgments,  the  anchoring  and  adjustment  heuristic 
(Hogarth,  1987) ,  could  produce  different  subjective  probabilities 
for  real  and  hypothetical  situations.  If  people  judge  the 
probability  of  a  hypothetical  case  by  producing  a  careful 
assessment  of  the  given  situation,  anchoring  on  that  assessment, 
and  adjusting  it  insufficiently  in  response  to  a  hypothetical 
change  in  the  situation,  then  the  same  situation  may  receive 
different  probability  assessments  if  called  "hypothetical"  or 
"actual" . 

The  knowledge  that  enables  quick  understanding  of  a  battlefield 
situation  is  pres  ’mably  acquired  with  years  of  experience.  This 
has  been  character,  .-.zed  as  "intuition",  sometimes  represented  as  a 
form  of  rapid  pattern  recognition  (Dreyfus  and  Dreyfus,  1986; 
Klein,  1989)  and  other  times  as  an  averaging  of  the  impacts  of 
multiple  inputs  (Hammond,  Hamm,  Grassia,  and  Pearson,  1987) . 
Whatever  the  details  of  the  process,  we  would  expect  that  the 
subjective  probability  and  importance  judgments  of  people  who 
have  had  experience  with  military  command  and  control  would  be 
more  accurate  than  the  judgments  of  people  with  no  experience, 
such  as  college  students.  For  comparison  purposes,  therefore,  the 
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study  was  replicated  with  college  students,  including  some  in  an 
Army  ROTC  program. 

In  summary,  this  study  is  concerned  with  three  aspects  of 
people's  judgments  about  hypothetical  military  decision 
situations.  The  following  predictions  are  made: 

1.  Probability  judgments. 

a.  The  judged  probability  of  an  evant  in  a  battlefield 
situation  (such  as  the  success  of  a  mission)  will 
vary  with  changes  in  the  facts  of  the  situation, 
but  it  will  not  vary  as  a  function  of  variations  in 
the  mood  of  the  presentation  that  do  not  carry  any 
information  content. 

b.  The  judged  probability  of  an  event  will  not  depend 
on  whether  the  event  is  described  as  "given"  or  as 
a  hypothetical  variant  of  the  given  situation. 

2.  Relative  importance  judgments:  In  the  absence  of 
instructions  to  make  global  relative  importance  judgments, 
people's  relative  importance  judgments  will  be  local 
rather  than  global.  Thus,  changes  in  the  situation  may 
induce  shifts  in  the  relative  importance  of  factors  that 
are  related  to  the  change. 

a.  Changes  in  the  facts  of  the  situation  may  induce 
shifts  in  relative  importance,  so  that  factors 
related  to  success  or  failure  will  be  judged  as 
more  important  when  the  facts  make  success  or 
failure,  respectively,  more  likely.  Such  changes 
would  be  legitimate  with  a  local,  but  not  with  a 
global,  definition  of  relative  importance. 

b.  Changes  in  the  mood  of  the  presentation  may  induce 
such  shifts,  so  that  factors  that  are  related  to 
the  dominant  mood  of  the  presentation  will  be 
judged  as  more  important.  Such  changes  would  not  be 
legitimate  with  either  a  local  or  a  global 
definition  of  relative  importance. 

3.  Experience:  Those  with  experience  in  military  decision 
making  will  have  more  appropriate  probability  and  relative 
importance  judgments  than  those  with  no  experience. 

Procedure 

The  stimulus  materials  consisted  of  a  typed  booklet,  including 
problem  descriptions  and  questions,  which  the  subjects  read  and 
responded  to  at  their  own  pace. 

Subjects.  This  questionnaire  was  sent  by  mail  to  688 
correspondence  students  of  the  U.S.  Army  Command  and  General 
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staff  College  (CGSC) ,  under  a  cover  letter  from  the  commander  of 
the  School  of  Corresponding  Studies  (SOCS)J  All  subjects 
received  a  postcard  reminder  after  approximately  one  month.  The 
booklet  was  also  delivered  to  the  mailboxes  of  100  resident  CGSC 
students  at  Fort  Leavenworth,  under  a  cover  letter  from  the 
director  of  the  Office  of  Evaluation  and  a  hand  written  note  from 
their  instructor.^  A  third  group  was  130  introductory  psychology 
students  at  the  University  of  Colorado,  who  did  the  procedure  in 
a  group  setting  under  the  eye  of  a  research  assistant.^  The  final 
group  was  24  Army  ROTC  students  at  the  University  of  Colorado.^ 

In  order  to  assure  that  the  CGSC  students  had  completed  the 
Tactics  course,  which  was  deemed  pertinent  background  for  the 
problems,  the  questionnaire  was  mailed  only  to  correspondence 
students  who  had  completed  at  least  Phase  III  of  the  self-paced 
curriculum,  and  it  was  given  to  the  resident  students  after  the 
Tactics  section  was  completed. 

Among  the  correspondence  students,  the  questionnaire  was  mailed 
to  soldiers  in  branches  most  likely  to  have  had  experience  and 
training  in  battlefield  situations:  armor,  artillery,  aviation, 
engineers,  and  infantry.  It  was  sent  to  students  from  three 
classes  of  service;  active  airmy,  national  guard,  and  reserves. 
There  were  a  number  of  different  versions  of  the  questionnaire. 
These  were  allocated  to  the  target  groups  in  bundles  (e.g.,  2 
bundles  were  sent  to  students  in  the  infantry  branch  of  the 
national  guard) ,  so  that  the  same  number  of  each  questionnaire 
went  to  each  type  of  subject.  The  overall  return  rate  from  the 
correspondence  students  did  not  differ  significantly  between 
branch  or  service  class.  The  CGSC  resident  students  were  not 
selected  on  the  basis  of  branch,  and  so  officers  from  all 
branches  were  included. 

There  were  two  mailings,  to  different  subsets  of  the  list  of 
qualifying  CGSC  correspondence  students.  One  condition  from  each 
problem  (Helicopter  Neutral,  and  River  Objective  Success;  see 
below)  was  included  in  the  first  mailing,  and  the  remainder  in 
the  second.  Also  in  the  second  mailing  were  questionnaires  which 
gave  outcome  information  for  the  same  problems,  reported  in  Hamm 
(1990d).  Return  rate  was  higher  for  the  questionnaires  of  the 
present  study  that  were  in  the  second,  fall  mailing  (42%  of  168) 
than  the  first,  summer  mailing  (29%  of  520) ,  presumably  because 
of  the  time  of  year.  Seventy-two  of  100  questionnaires  were 
returned  by  the  resident  CGSC  students.  Among  the  293  responding 
military  officers,  3.7%  were  lieutenant  colonels,  72.1%  majors, 
and  24.1%  captains.  The  mean  age  was  38.1  (active  army:  35.9; 


1.  The  assistance  of  Dr.  Ernest  Lowden  in  identifying  this  population  and  of  Captain  Hike  Spry  and  LTC 
William  Brethorst  of  SOCS  in  making  them  available  is  acknowledged. 

2.  Thanks  again  to  Dr.  Lowden  and  to  the  instructors. 

3.  Thanks  to  Susie  Donaldson. 


4.  Thanks  to  Major  Steve  Whitworth  of  the  Arnny  ROTC  program  at  the  University  of  Colorado. 


national  guard:  39.9;  reserves:  39.5).  Questionnaires  were 
collected  from  all  participating  University  of  Colorado  students 
though  a  small  proportion  of  responses  were  not  usable. 

Mailed  questionnaires  were  used  with  the  military  subjects  in 
order  to  get  a  large  number  of  subjects.  The  low  return  rate  was 
anticipated.  Although  there  is  the  danger  that  selective  response 
might  produce  a  sample  whose  judgments  on  these  problems  are  not 
representative  of  that  of  the  general  population  of  Army 
officers,  this  was  considered  unlikely  and  a  risk  worth  taking. 
Comparison  between  the  CGSC  resident  students  (72%  response  rate) 
and  the  correspondence  students  (33%  response  rate)  can  reveal 
differences  that  might  suggest  an  effect  of  self-selected 
membership  in  the  sample  on  the  results.  No  differences  were 
found  between  the  two  groups  in  the  judged  probability  of  success 
nor  the  judgments  of  relative  importance.  Further,  probability  of 
success  was  not  related  to  variables  that  index  experience  (age, 
education,  years  in  army,  months  in  various  command  and  staff 
positions) .  Therefore  the  military  officers  will  be  treated  in 
what  follows  as  one  homogeneous  group. 

Materials.  Two  problems  representing  situations  facing  a  military 
commander  were  used.  In  the  Helicopter  problem  (Appendix  A)  a 
Company  commander  must  respond  to  a  helicopter  heard  to  crash 
near  his  position.  The  mission  is  to  find  the  helicopter  and 
return  with  the  injured  or  prisoners.  In  the  River  problem 
(Appendix  B) ,  a  Battalion  commander  must  plan  an  attack  across  a 
river. ^  The  mission  is  to  take  a  hill  across  the  river  by  a 
particular  time.  The  CGSC  correspondence  students  received  both 
problems.  The  CGSC  resident  students  and  the  CU  students  received 
only  the  Helicopter  problem. 

Five  versions  of  each  of  the  problems  were  prepared  (Table  1) , 
varying  expectation  of  success.  Starting  with  the  Neutral  version 
(Helicopter  (HN)  in  Appendix  A,  and  River  (RN)  in  Appendix  B) , 
two  versions  varied  the  objective  threat  presented  by  the  enemy. 
The  text  that  was  varied  is  highlighted  in  the  appendices.  Table 
2  shows  the  text  that  was  substituted  to  produce  the  low  threat 
(high  expectation  of  success)  and  high  threat  (low  expectation  of 
success)  versions.  These  will  be  referred  to  as  the  "Objective 
Success"  (HOS  and  ROS)  and  "Objective  Failure"  (HOF  and  ROF) 
expectation  conditions,  and  can  be  compared  with  the  "Neutral" 
condition. 


5.  The  Helicopter  problem  is  a  modified  version  of  a  story  used  originally  by  Olmstead,  as  remembered  by 
Lloyd  Crumley.  The  River  problem  was  developed  from  a  suggestion  by  Stan  Halpin.  The  assistance  of  the 
people  at  the  ARI  Fort  Leavenworth  Field  Unit  (especially  Lloyd  Crunley,  Rex  Michel,  Major  Ed  Sullivan 
and  Captain  Doug  Litavec,  as  well  as  Jim  Lussier,  Sharon  Riedel,  and  Bob  Solick),  and  of  instructors  at 
CGSC,  Fort  Leavenworth,  in  making  these  problems  realistic  and  engaging,  is  gratefully  acknowledged. 
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Table  1 

Problem  versions  representing  Objective  and  Subjective 
manipulations  of  expectation  of  success. 


Problem 

Information 

Manipulated 

Expectation  of 
Failure  Neutral 

Success 

Success 

Helicopter 

Objective: 

Facts 

HOF 

HN 

HOS 

Subjective: 

Presentation 

Mood 

HSF 

HSS 

River 

Objective: 

Facts 

ROF 

RN 

ROS 

Subjective: 

Presentation 

Mood 

RSF 

RSS 

The  other  two  versions  of  each  problem  changed  the  mood  of  the 
situation  presentation,  varying  the  tone  but  keeping  the 
objective  information  the  same.  Starting  with  the  Neutral 
condition  (Appendices  A  and  B) ,  the  text  in  the  left  column  of 
Table  3  (for  the  Helicopter  problem)  or  Table  4  (for  the  River 
problem)  was  substituted  to  create  a  subjective  expectation  of 
failure  (HSF  and  RSF) ,  and  the  text  in  the  right  column  was 
substituted  to  create  a  subjective  expectation  of  success  (HSS 
and  RSS) .  Thus  the  mood  or  subjective  presentation  manipulation 
is  achieved  by  comparing  the  "Subjective  Failure",  "Neutral",  and 
"Subjective  Success"  conditions:  HSF  versus  HN  versus  HSS,  and 
RSF  versus  RN  versus  RSS. 
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Table  2 

Text  conveying  Objective  manipulations  of  reader's  expectations 
concerning  mission  outcome. 

Helicopter  problem. 


High  objective  enemy  strength  (Objective  Failure  condition,  HOF) : 
"There  has  been  particularly  heavy  enemy  patrolling  and 
reconnaissance  activity  in  this  sector  over  the  last  several 
days.  Intelligence  thinks  that  an  attack  may  be  imminent,  and 
warns  that  the  company  should  be  alert  for  troop  infiltrations 
and  attacks  on  key  installations." 

Medium  objective  enemy  strength  (Neutral  condition,  HN) :  "There 
has  been  little  enemy  activity  in  this  sector,  other  than  probes 
and  occasional  contacts,  over  the  last  several  days.  Intelligence 
does  not  think  that  an  attack  is  imminent,  but  warns  that  the 
company  should  be  alert  for  reconnaissance  patrols  and  probes." 

Low  objective  enemy  strength  (Objective  Success  condition,  HOS) ; 
"There  has  been  very  little  enemy  activity  in  this  sector 
recently.  Although  there  are  usually  some  reconnaissance  patrols 
and  probes,  none  have  been  observed  over  the  last  several  days. 
Intelligence  thinks  that  some  of  the  enemy  forces  may  have  been 
moved  to  the  north  in  support  of  a  campaign  in  that  area." 


River  Problem. 


High  objective  enemy  strength  (Objective  Failure  condition,  ROF) : 
"Intelligence  reports  that  the  enemy  . . .  have  been  attrited  to 
90%  strength." 

Medium  objective  enemy  strength  (Neutral  condition,  RN) : 
"Intelligence  reports  that  the  enemy  . . .  have  been  attrited  to 
70%  strength." 

Low  objective  enemy  strength  (Objective  Success  condition,  ROS) : 
"Intel] igence  reports  that  the  enemy  . . .  have  been  attrited  to 
50%  strength." 


10 


Table  3 

Text  conveying  subjective  manipulations  of  reader's  expectations 
concerning  mission  outcome,  Helicopter  problem. 


Subjective 

Failure 

Condition  (HSF) 


Neutral 
Condition  (HN) 


Subjective 

Success 

Condition  (HSS) 


1.  Nevertheless,  Captain  (Nothing  said. 

Smith  reminds  himself  Location:  after 

frequently  that  the  enemy  "...last 
is  a  viciously  efficient  several 

military  force  that  has  days.") 

surprised  U.S.  forces  a 
number  of  times  in  this 
conflict. 


Captain  Smith  and 
his  men  joke  freq 
uently  about  the 
enemy ' s  legendary 
inefficiency. 


. . .thick  fog. . . 
intermittent 
gentle  rain... 


2.  ...thick  ominous 
fog. . . intermittent 
light  rain. . . 

3.  Good  luck  —  you 
are  going  to  need 
it! 


4.  Capt.  Smith  notices 
that  Lt.  Wilson  has 
doubts  about  the 
mission  and  is  not 
eager  to  lead  the 
patrol  himself. 
Wilson  decides. . . 


. . .thick  fog. . . 
intermittent 
light  rain. . . 

(Nothing  said. 
Location:  after 
"from  the  survivors 
or  the  wreckage.") 

Capt.  Smith  notices 
that  Lt.  Wilson  is 
eager  to  lead  the 
patrol  himself,  but 
Wilson  decides... 


You  should  be  able  to 
pull  it  off  successfully 
if  you  get  a  move  on! 


Capt  Smith  notices  that 
Lt.  Wilson  is  confident 
about  the  mission  and  is 
eager  to  lead  the  patrol 
himself,  but  Wilson 
decides. . . 


Jones,  whose  military 
bearing  accentuates 
his  rugged  appearance, 
has  led. . . 


As  they  take  off,  Sgt. 
Jones  is  heard  to 
remark,  "This  will  be 
a  piece  of  cake." 


5.  Jones,  whose  sloppy  Jones  has  led... 
uniform  and  chronic 

frown  reinforce  the 
impression  that  he 
has  let  himself  get 
out  of  shape,  has  led... 

6.  As  they  take  off,  (Nothing  said. 

Sgt.  Jones  is  heard  Location:  at  end.) 
to  remark,  "It  prob¬ 
ably  came  down  right 

by  an  enemy  patrol . " 
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Table  4 

Text  conveying  subjective  manipulations  of  reader's  expectations 
concerning  mission  outcome,  Piver  Problem. 


Subjective 

Failure 

Condition  (RSF) 


Neutral 
Condition  (RN) 


Subjective 

Success 

Condition  (RSS) 


1.  Because  of  the  late 
hour  he  moves  slow¬ 
ly,  with  the  ten¬ 
sion  showing  in  his 
face. 


(Nothing  said. 
Location:  after 
"plans  for  the  next 
day's  battle.") 


2.  ...several  hours,  ...several  hours, 
and  can  be  assumed 
to  be  vigorously 
preparing  their  defenses. 


3 .  Rubel  is  anxious 
that  they  might 
attack  here  tonight 
or  tomorrow. 

4 .  . . .they  will  be 
able  to  bring  high¬ 
ly  effective  fire 

5.  He  hopes  this  will 
make  it  more  diff¬ 
icult  for  them 


They  are  not  likely 
to  attack  here 
tonight  or 
tomorrow. 

. . .they  will  be 
able  to  bring  fire 


This  should  make  it 
more  difficult  for 
them 


6.  Companies  A  and  D  Companies  A  and  D 
will  probably  be  will  cross  ford  X 
able  to  cross  ford  X 


7.  ...shift  target...  ...shift  target... 
Hill  434.  Hopefully  Hill  434.  This  will 
this  will  prevent  prevent 

8.  Assuming  the  plan  The  artillery  will 
has  worked  thus  far,  shift  to 

the  artillery  will  suppressing 
shift  to  suppressing 

9.  LTC  Rubel  heads  off  (Nothing  said. 

to  sleep  wondering  Location:  at  end.) 
whether  the  plan 
will  work. 


Despite  the  late  hour  he 
moves  briskly,  projecting 
his  total  commitment  and 
dedication. 


...several  hours,  and  can 
be  assumed  to  be 
exhausted. 


Rubel  is  confident  that 
they  will  not  attack 
here  tonight  or 
tomorrow. 

...they  will  probably 
be  able  to  bring 
some  fire 

He  is  confident  this 
will  make  it  more 
difficult  for  them 

Companies  A  and  D  will 
be  able  to  cross  ford  X 


. . .shift  target. . . 
Hill  434.  This  will 
prevent 

The  artillery  will 
shift  to 
suppressing 


LTC  Rubel  heads  off 
to  sleep  quite 
confident  that  the 
plan  will  work. 


After  each  problem,  the  subject  was  asked  to  (1)  state  the 
probability  that  the  mission  would  succeed,  (2)  revise  the  plan 
to  make  it  better,  and  (3)  state  the  probability  that  his  or  her 
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revised  plan  would  succeed.  The  subject  was  asked  to  revise  the 
plan  in  order  to  assure  careful  consideration  of  the  problem.  The 
ROTC  students,  who  participated  in  a  group  setting,*  put 
considerable  effort  into  this  task,  writing  many  paragraphs, 
while  the  introductory  psychology  students  typically  wrote  only  a 
few  sentences.  The  amount  of  writing  in  the  military  officers' 
plans,  which  were  produced  in  individual  settings,  varied  widely. 
Then  the  subject  (4)  rated  the  relative  importance  of  eight 
factors  in  the  story,  using  two  different  methods.  After  the 
second  problem,  the  subject  (5)  rated  the  two  relative  importance 
assessment  methods  on  seven  questions.  (6)  said  which  of  the  two 
methods  he  or  she  preferred,  and  (7)  filled  out  a  page  of 
background  information.  The  procedure  could  take  from  10  minutes 
to  an  hour,  depending  on  the  amount  of  effort  the  subject  put 
into  it  and  on  whether  there  were  1  or  2  problems.  The  method 
preferences  and  evaluations  are  presented  in  Hamm  (1990b) . 

The  factors  whose  relative  importances  were  assessed  in  the 
Helicopter  problem  were  either  unknown  aspects  of  the  situation, 
or  aspects  of  the  commander's  hurried  plan:  the  fog  in  the  valley 
where  the  crash  occurred,  the  actual  location  of  the  crash,  what 
the  enemy  is  doing,  the  decision  to  send  one  patrol  rather  than 
two,  the  decision  to  send  a  small  patrol,  the  choice  of  the  most 
experienced  man  to  lead  the  patrol,  the  choice  of  the  most  direct 
route  to  the  estimated  crash  site,  and  the  decision  not  to 
request  aerial  reconnaissance  before  sending  out  the  patrol.  The 
factors  in  the  River  problem  were  assumptions  that  were  made  in 
the  problem;  that  the  enemy  has  a  given  strength,  that  friendly 
artillery  support  is  available,  that  the  river  can  be  forded, 
that  vehicles  can  pass  over  the  ground  on  the  far  side  of  the 
river,  that  the  enemy  does  not  occupy  that  ground,  that  the  enemy 
has  not  completed  preparation  of  defensive  positions,  that  there 
will  be  no  friendly  air  support,  and  that  there  will  be  no  enemy 
air  support. 

Relative  importance  judgment  methods.  Three  different  methods  for 
measuring  relative  importance  were  used  in  this  study;  Yes/No 
(subject  indicated  whether  each  factor  was  important  or  not) ; 
Rating  (subject  used  a  continuous  numerical  scale) ;  and 
Probability  Change  (subject  considered  a  change  from  the  current 
level  of  each  dimension  and  said  what  the  probability  of  mission 
success  would  be  if  that  were  the  case,  all  other  factors  being 
unchanged) . 

Instructions  for  each  method  started  with  a  general  definition  of 
importance,  in  the  context  of  the  particular  problem.  For  the 
River  problem,  "The  success  of  LTC  Rubel's  attack  plan  depends  on 
several  important  factors."  The  instructions  continued  as  shown 
in  the  following  paragraphs. 


6.  The  students'  professor  and  program  commander  observed  the  session,  which  followed  immediately  after  a 
talk  about  their  future  in  the  Army  and  about  where  among  them  the  "cut"  would  fall  this  year. 
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Yes/No.  "Please  consider  the  following  list,  and  judge  whether 
each  factor  is  important  in  determining  the  success  of  the 
attack."  [Each  of  the  8  factors  given  above  was  listed,  with 
"Yes/No"  placed  to  its  left.] 

Rating.  "Please  consider  the  following  list,  and  rate  the  factors 
according  to  their  importance.  Pick  a  fairly  unimportant  factor, 
and  give  it  a  score  of  '10'.  Then  give  the  other  factors  numbers 
that  reflect  how  important  they  are,  in  relation  to  that  first 
factor.  Thus,  you  might  give  a  very  important  factor  a  '200' 
because  it  is  20  times  as  important  as  the  first,  or  you  might 
give  an  unimportant  factor  a  '5'  because  it  is  only  half  as 
important  as  the  first  factor.  You  can  use  fractions  and  can  go 
as  close  to  0  as  you  want,  or  as  high  as  you  want."  [Each  factor 
was  listed,  with  a  blank  to  its  left.] 

Probability  Change.  "Please  consider  the  following  list,  and 
estimate  what  the  probability  Oi.  attack  success  would  be  if  the 
factor  were  to  change  as  described.  Please  give  specific 
probabilities.  Each  of  the  changes  should  be  considered 
independently,  leaving  all  other  features  as  described 
originally.  Refer  to  the  'Plan  Evaluation*  question  to  review 
your  estimate  of  the  probability  that  the  attack  will  succeed  in 
taking  Hill  434  by  1200  given  current  conditions  and  LTC  Rubel's 
original  plan." 

In  this  method  the  factors  were  presented  in  a  unique  manner.  A 
specific  alternative  value,  different  from  that  given  in  the 
original  problem  presentation,  was  named.  For  example,  in  the 
River  problem  it  is  assumed  that  the  attack  will  have  artillery 
support:  "two  batteries  of  155s...  have  been  made  available  to 
the  battalion  during  the  morning  hours...".  The  judgment  of  the 
relative  importance  of  friendly  artillery  is  elicited  using, 

"What  would  be  the  probability  of  attack  success  if  the  brigade 
were  not  able  to  provide  the  battalion  with  full  artillery 
support  during  the  attack?"  The  questions  for  each  of  the  factors 
are  given  in  Appendices  C  (Helicopter  problem)  and  D  (River 
problem) . 

In  the  River  problem,  the  Objective  manipulation  varied  the  enemy 
strength  (50%,  70%,  and  90%  strength  for  the  Objective  Success, 
Neutral,  and  Objective  Failure  conditions,  respectively).  The 
question  used  in  the  Probability  Change  method  to  assess  the 
importance  of  the  enemy  strength  factor  was  "What  would  be  the 

probability  of  attack  success  if  the  enemy  in  fact  were  at  _ % 

strength?",  with  values  of  75%,  90%,  and  70%  offered  as 
hypothetical  alternatives  for  50%,  70%,  and  90%,  respectively. 
Because  this  variation  in  method  makes  it  difficult  to  interpret 
the  effects  of  changes  in  enemy  strength  on  the  resulting 
importance  ratings,  the  Probability  Change  method  will  not  be 
considered  in  discussing  the  results  of  manipulating  enemy 
strength  in  the  Objective  conditions  of  the  River  problem. 
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Design.  Some  of  the  CGSC  correspondence  student  subjects  were 
part  of  a  separate  study  that  focussed  on  comparing  the  methods 
for  measuring  relative  importance  (Hamm,  1990a,  1990b) .  In  the 
design  of  this  study,  the  order  of  the  problems  (River  or 
Helicopter  first)  was  crossed  systematically  with  which  two 
methods  (drawn  from  a  set  of  five)  were  used.  Thus  subjects  who 
did  the  Neutral  condition  for  the  Helicopter  problem  (HN  in  Table 
1)  also  did  the  Objective,  Success  condition  for  the  River 
problem  (ROS) .  And  they  might  have  used  any  one  or  two  of  the 
three  relative  importance  assessment  methods  whose  results  are 
reported  here  (see  below) .  They  used  the  same  two  methods,  in  the 
same  order,  on  each  of  the  two  problems. 

The  remainder  of  the  CGSC  correspondence  students  were  part  of  a 
second  study  in  which  each  of  the  stories  was  varied.  The 
Helicopter  problem  was  always  presented  first.  A  particular 
condition  on  the  Helicopter  problem  was  always  presented  with  a 
particular  condition  on  the  River  problem.  Thus,  when  the 
Helicopter  problem  had  the  Subjective,  Failure  (HSF)  condition, 
it  was  followed  by  the  Neutral  condition  of  the  River  problem 
(RN) .  Other  pairs  were:  (HSS,  ROF) ,  (HOF,  RSS) ,  and  (HOS,  RSF) . 
Each  subject  did  a  "subjective"  (or  neutral)  variant  of  one 
problem  and  an  "objective"  (or  neutral)  variant  of  the  other. 
Crossed  with  the  problem  variations,  subjects  in  this  study 
assessed  the  relative  importance  of  the  factors  using  two  of 
these  three  methods;  Yes/No,  Rating,  Probability  Change.  The  two 
methods  were  always  presented  in  the  order  just  listed.  The  fixed 
orders  (Helicopter  before  River,  and  Yes/No  before  Rating  before 
Probability  Change)  were  selected  so  that  the  easier  problem  or 
procedure  was  done  first. 

The  CGSC  resident  students  and  the  University  of  Colorado 
students  were  given  one  of  the  five  versions  of  the  Helicopter 
problem.  Their  relative  importance  judgments  were  assessed  using 
the  Rating  and  then  the  Probability  Change  method. 

Results 

Probability  judgments.  Subjects  estimated  the  probability  of 
mission  success  after  reading  the  description  of  the  situation. 
There  is  no  correct  probability  to  compare  their  answers  to, 
because  the  situations  are  hypothetical  and  incompletely 
described.  However,  subjects'  probabilities  were  expected  to 
change  in  response  to  changes  in  the  facts  of  the  situation, 
i.e.,  the  objective  description  of  the  enemy's  strength  (HOF,  HN, 
and  HOS;  ROF,  RN,  and  ROS  in  Table  1),  but  not  to  change  in 
response  to  changes  in  the  mood  of  the  presentation  of  the 
situation  (HSF,  HN,  and  HSS;  RSF,  RN,  and  RSS) . 

The  CGSC  students'  mean  subjective  probabilities  for  the  variants 
of  the  Helicopter  problem  are  the  bottom  two  curves  in  Figure  1 
(the  university  students'  curves  will  be  discussed  below).  The 
judgments  were  highly  variable:  the  mean  within-condition 
standard  deviation  for  the  military  officers  on  the  Helicopter 
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problem  was  .186.  The  subjectis*  judgments  of  probability  of 
mission  success  did  not  change  monotonically  with  the 
manipulation  of  the  facts  about  the  enemy  in  the  Objective 
scenario  variants  (solid  line) .  The  officers  thought  success  was 
more  likely  in  the  Neutral  condition  (HN;  mean  probability  =  .57, 
n  =  166)  than  in  either  the  Success  (HOS;  .54,  n  =  29)  or  Failure 
(HOF;  .49,  n  =  34)  conditions.  Overall,  these  fell  short  of 
differing  significantly  (F(2,227)  =  1.9,  p  =  .15);  the  Neutral 
condition  (HN)  was  judged  marginally  significantly  more  likely  to 
succeed  than  the  Failure  condition  (HOF;  t(198)  =  1.9,  p  =  .060). 


. Universi iy 

Subject ive 

- Universi ty 

Qb  3  ec  1 1 ve 

. CG5C 

Sub)ect ive 

- CG5C 

Objective 


Relative  Expectation  Success 
Established  in  the  Scenario 


Figure  1.  Judged  probability  of  mission  success,  Helicopter 
problem,  military  and  college  students. 

The  probabilities  of  the  Subjective  variants  of  the  Helicopter 
problem  (dotted  line  in  Figure  1)  varied  as  a  direct  function  of 
the  mood  of  the  presentation.  Subjects  judged  the  Success  mood 
presentation  (HSS)  to  have  a  .58  probability  of  success  (n  =  29); 
the  Neutral  presentation  (HN)  a  .57  probability  (as  above;  n  = 
166) ;  and  the  Failure  mood  presentation  (HSF)  to  have  a  .47 
probability  of  success  (n  =  29).  These  differed  significantly 
(F(2,222)  =  3.1,  p  =  .046).  The  judged  probability  of  the 
Subjective  Failure  condition  (HSF)  was  significantly  less  than 


16 


the  Subjective  Success  condition  (HSS;  t(56)  =  2.5,  p  =  .014)  and 
than  the  Neutral  condition  (HN;  t(193)  =  2.4,  p  =  .020). 


The  probabilities  of  success  for  Objective  changes  in  the  River 
problem  revealed  a  similar  pattern  to  those  on  the  Helicopter 
problem  (solid  line  in  Figure  2) .  Again,  the  judged  probability 
of  success  was  not  a  monotonic  function  of  the  enemy  strength. 
When  the  enemy  was  at  50%  strength  (Objective  Success  condition, 
ROS)  the  mean  probability  of  success  was  .61  (n  =  150);  with  70% 
enemy  strength  (Neutral  condition,  RN) ,  probability  of  success 
was  .70  (n  =  14);  and  with  90%  enemy  strength  (Objective  Failure 
condition,  ROF) ,  probability  of  success  was  .58  (n  =  16). 


Probability  of  Success 


Sub  j  ec  1 1 ve 
■Objective 


Relative  Expectation  of  Success 
Established  in  the  Scenario 


Figure  2.  Judged  probability  of  mission  success.  River  problem, 
CGSC  correspondence  students. 


Overall  this  pattern  was  not  conventionally  significant  (F(2,178) 
=  2.2,  p  =  .12),  although  the  N*>utral  condition  (RN)  was  judged 
significantly  more  likely  than  the  Success  condition  (ROS;  t(162) 
=  -1.9  (counter  expectations),  p  =  .066)  and  than  the  Failure 
condition  (ROF;  t(28)  =  2.4,  p  =  .025).  Unlike  in  the  Helicopter 
problem,  where  the  facts  about  the  enemy  were  manipulated  by 
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changing  a  number  of  verbal  descriptions,  the  only  aspect  of  the 
scenario  varied  in  the  Objective  conditions  of  the  River  problem 
was  the  number  representing  the  enemy  strength.  The  lack  of 
sensitivity  to  this  important  factor  is  surprising. 

The  probabilities  of  the  Subjective  variants  of  the  River  problem 
(dotted  line  in  Figure  2)  revealed  the  same  pattern  as  the 
Objective  variants:  success  was  judged  most  likely  in  the  middle 
or  Neutral  (RN)  condition  (.70;  as  above,  n  =  14).  The  mean 
probability  of  mission  success  in  the  Subjective,  Success 
condition  (RSS)  was  .64  (n  =  18)  and  in  the  Subjective,  Failure 
condition  (RSF) ,  .62  (n  =  18).  The  differences  among  these  and 
the  Neutral  condition  are  not  statistically  significant  (F(2,48) 

=  0.9,  p  =  .40) . 

Probabilities  for  the  szuae  situation  when  "given"  versus 
"hypothetical".  The  Probability  Change  measure  of  relative 
importance  asks  subjects  to  judge  the  probability  of  mission 
success  given  a  specified  change  in  a  factor.  In  the  River 
problem,  one  of  the  factors  whose  importance  was  measured  was  the 
strength  of  the  enemy.  Thus,  for  the  Neutral  condition  (RN)  the 
given  enemy  strength  was  70%  and  the  subject  was  asked,  "What 
would  be  the  probability  of  attack  success  if  the  enemy  in  fact 
were  at  90%  strength?"  However,  the  Objective  manipulation  varied 
the  enemy  strength  given  in  the  problem  to  90%  for  the  Objective 
Failure  condition  (ROF) .  Consequently  it  was  necessary  to  have 
the  subjects  consider  a  different  alternative  enemy  strength  in 
the  latter  condition,  and  70%  was  used.  This  gives  an  opportunity 
to  test  whether  people  judge  an  outcome  to  have  the  same 
probability  when  the  situation  is  "given"  or  "hypothetical". 

Subjects  in  the  Objective  Failure  condition  (ROF)  who  used  the 
Probability  Change  method  were  told  that  the  enemy  was  at  90% 
strength  and  asked  to  consider  the  probability  of  success  if  the 
enemy  had  been  at  70%  strength.  In  contrast,  subjects  in  the 
Neutral  condition  (RN)  were  told  that  the  enemy  was  at  70% 
strength  and  asked  to  consider  the  probability  of  success  if  the 
enemy  had  been  at  90%  strength.  Each  group,  therefore,  provides 
judgments  of  the  probability  of  mission  success  when  the  enemy 
strength  is  at  two  levels:  70%  and  90%.  The  difference  is  that 
one  of  these  strengths  is  given  as  part  of  the  original 
presentation  of  the  scenario,  while  the  other  is  considered  as  a 
hypothetical  change  from  the  original  scenario.  If  subjects  judge 
"hypothetical"  and  "given"  scenarios  identically,  each  scenario 
with  a  particular  enemy  strength  should  be  assigned  the  same 
probability.  On  the  other  hand,  if  they  use  a  process  in  which 
less  attention  is  given  to  the  hypothetical  information,  then  the 
probability  judgment  may  differ.  Such  a  process,  for  example, 
might  involve  "anchoring"  on  the  probability  judgment  for  the 
given  information  and  then  "adjusting"  the  probability  slightly 
in  the  direction  implied  by  the  hypothetical  change  in  the 
situation. 
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The  pattern  of  results  is  shown  in  Figure  3.  Subjects  in  the 
Neutral  condition  (RN;  n  =  10^)  judged  the  probability  of  success 
when  enemy  strength  was  70%  to  be  .71.  When  asked  to  consider  the 
hypothetical  situation  in  which  the  enemy  strength  was  90%,  they 
changed  their  probability  of  mission  success  to  .51.  Subjects  in 
the  Objective  Failure  condition  (ROF;  n  =  10)  also  estimated  the 
probability  of  mission  success  to  be  .51  if  the  enemy  was  at  90% 
strength,  but  they  increased  it  only  to  .60  (rather  than  to  .71) 
in  the  hypothetical  case  of  70%  strength.  Thus,  when  the  subject 
was  given  information  that  the  enemy  was  strong  (90%  strength) 
and  then  considered  a  hypothetical  situation  in  which  the  enemy 
was  weak  (70%  strength) ,  this  weak  enemy  was  judged  to  be  more  of 
a  threat  to  the  success  of  the  mission  than  when  it  was  given  in 
the  original  description  of  the  situation.  This  suggests  that 
subjects  may  be  anchoring  on  the  probability  assessments  they 
make  for  the  originally  given  information,  and  adjusting  these 
probabilities  insufficiently  when  asked  to  consider  hypothetical 
changes  in  the  situation. 

A  repeated  measures  analysis  of  variance  tests  the  statistical 
significance  of  this  effect.  Mission  success  was  judged  more 
likely  when  enemy  was  weak  than  strong  (F(l,18)  =  35.4,  p  = 

.000),  and  the  situation  originally  considered  had  more  influence 
on  the  average  judgment  than  the  situation  hypothetically 
considered  (F(l,18)  =  4.8,  p  =  .042).  However,  the  between- 
subject  t-test  between  the  two  cells  representing  the  judgments 
of  the  70%  strength  enemy  when  this  information  was  given  in  the 
problem,  versus  considered  hypothetically,  was  not  statistically 
significant  (t(18)  =  1.4,  p  =  .17).  Therefore,  these  results 
suggest  but  do  not  establish  that  officers  anchor  and 
insufficiently  adjust,  i.e.,  that  they  judge  hypothetical  changes 
in  a  situation  differently  from  how  they  judge  the  focal 
situation. 

Relative  importance  judgments.  For  each  problem,  subjects  made 
relative  importance  judgments  concerning  each  of  8  factors.  The 
judgments  were  made  using  three  different  methods  (Yes/No, 

Rating,  and  Probability  Change;  each  subject  used  two  of  the 
three®)  .  For  comparison  purposes,  the  judgments  are  translated  to 
a  common  numerical  scale. 


7.  Only  subjects  who  used  the  Probability  Change  method  are  included  in  this  comparison,  so  the  Ms  are  less 
than  in  the  results  presented  above. 

8.  As  explained  above,  some  of  the  subjects  who  did  the  Helicopter  Neutral  (HN)  and  the  River  Objective 
Success  (ROS)  problems  did  less  than  two  of  these  three  methods,  because  subjects  in  this  group  used  two 
out  of  a  set  of  five  methods;  results  with  the  other  methods  are  not  considered  here. 
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Probability  ol  Success 


Anchor  9G°^ 
Probe  70'': 
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Probe  90'/. 


S t  reng t h  o f  the  Enemy 
in  the  Situation  Judged 


Figure  3.  Probability  of  success  for  two  levels  of  enemy 
strength,  as  a  function  of  whether  the  level  is  given  in  the 
original  situation  or  considered  as  a  hypothetical  alternative. 

To  do  this,  the  subjects'  judgments  are  rescaled  onto  a  numerical 
scale,  and  then  the  numerical  scales  from  the  different  methods 
are  normalized  to  produce  a  common  scale.  Yes/no  responses  were 
rescaled  as  follows:  Yes  =  l.  No  =  0.  Rating  responses  did  not 
need  to  be  rescaled  prior  to  normalization.  For  the  probability 
change  method,  importance  was  measured  by  the  absolute  value  of 
the  difference  between  the  original  estimate  of  the  plan's 
probai-ility  of  success  and  the  revised  estimate  given  the  changed 
factor. 

Normalization  was  accomplished  by  dividing  each  factor's 
numerical  importance  measure  by  the  sum  over  all  8  factors.  This 
produced  a  number  ranging  between  0  and  1  for  each  factor,  with  a 
total  of  1  over  all  8  factors  and  hence  an  expected  value  of 
.125.  Normalization  was  done  only  for  those  subjects  who  gave  an 
answer  for  all  8  factors;  otherwise  the  subject's  data  on  the 
measure  were  not  further  used.  Finally,  summary  measures  of 
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relative  importance  can  be  created  by  averaging  the  normalized 
scores  for  several  methods. 

On  the  rating  method  four  of  the  CGSC  subjects  appeared  to  have 
rank  ordered  the  factors  -  they  used  each  of  the  numbers  from  1 
to  8.  Because  of  the  ambiguity  —  a  low  rank  is  important,  while 
a  high  rating  is  important  —  it  was  necessary  to  drop  these 
cub j ec^s '  ratings . 

On  the  River  problem,  with  the  probability  change  importance 
judgment  method,  the  question  used  to  assess  the  importance  of 
the  enemy's  strength  was  varied  between  Objective  conditions 
because  the  actual  enemy  strength  itself  was  varied  (see  above) . 
This  variation  makes  it  inappropriate  to  use  the  probability 
change  measure  for  comparing  the  Objective  conditions  on  the 
River  problem. 

Local  versus  global  judgments  of  relative  importance.  The  Yes/No 
and  Rating  methods  can  be  used  either  locally  or  globally.  In 
contrast,  the  Probability  Change  method  is  explicitly  local. 
Therefore  only  the  Yes/No  and  Rating  methods  can  be  used  to  test 
whether  subjects  spontaneously  adopt  a  local  or  global 
interpretation. 

A  local  interpretation  of  relative  importance  is  suggested  if  the 
relative  importance  rating  for  a  factor  changes  when  the  level  of 
the  factor  in  the  particular  situation  changes.  Such  a  change 
would  be  inconsistent  with  a  global,  and  consistent  with  a  local, 
interpretation  of  relative  importance.  Lack  of  change  does  not 
disprove  a  local  interpretation,  although  it  is  more  consistent 
with  a  global  interpretation. 

The  Objective  conditions  explicitly  varied  the  level  or  value  of 
a  factor  in  the  situation  (see  Table  2) .  For  both  problems,  this 
factor  was  the  enemy  threat;  the  "enemy  acts"  factor  in  the 
Helicopter  problem,  and  the  "enemy  strength"  factor  in  the  River 
problem  (see  Appendices  C  and  D) .  The  judged  importance  of  the 
varied  factor  (mean  of  normalized  Yes/No  and  Rating  responses) 
changed  very  little  in  the  Helicopter  problem  (Objective,  Enemy 
Acts  factor:  thin  solid  line  [HOF  versus  HN  versus  HOS]  in  Figure 
4;  F(2,173)  =  0.2,  p  =  .81).  Change  was  insignificant  in  the 
River  problem  as  well  (Objective,  Enemy  Strength  factor:  thin 
solid  line  [ROF  versus  RN  versus  ROS]  in  Figure  5;  F(2,105)  = 

0.5,  p  =  .59).  This  suggests  that  people  use  relative  importance 
in  a  global  sense. 
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Figure  4.  Relative  importance  of  specific  factors.  Helicopter 
problem,  objective  and  subjective  conditions,  military  officers. 

The  purpose  of  the  Subjective  manipulations  was  to  change 
expectation  through  manipulation  of  mood  and  characters' 
feelings.  It  was  not  our  intent  to  manipulate  any  of  the  actual 
factors  in  the  situation.  However,  inspection  of  the  problem 
variants  shows  that  some  of  the  manipulation  sentences  might  be 
construed  as  changing  the  cue  values  of  some  of  the  factors. 
Specifically,  in  the  Helicopter  problem  changes  1  and  6  (see 
Table  3)  had  to  do  with  the  enemy,  and  changes  4  and  5  had  to  do 
with  the  choice  of  the  more  experienced  man  as  leader  of  the 
patrol.  In  the  River  problem,  changes  2  and  4  (in  Table  4)  had  to 
do  with  the  enemy's  strength,  change  6  with  the  commander's 
forces'  ability  to  ford  the  river,  and  change  8  with  the  power  of 
the  friendly  artillery.  Because  of  the  possibility  that  these 
factors  were  changed  by  the  Subjective  manipulations,  we  will 
check  to  see  whether  subjects'  judgments  of  the  relative 
importance  of  these  factors  was  affected.  A  change  would  support 
the  hypothesis  that  subjects  make  relative  importance  judgments 
in  a  local  manner.  But  because  the  manipulation  of  the  factors 
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was  minor,  if  there  were  no  change  it  would  provide  only  weak 
evidence  that  they  judge  relative  importance  globally. 
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Figure  5.  Relative  importance  of  specific  factors,  River  problem, 
objective  and  subjective  conditions,  military  officers. 

Subjects'  relative  importance  ratings  for  these  factors  did  not 
vary  significantly  as  an  effect  of  the  Subjective  manipulation  of 
the  expectation  of  success  through  the  mood  of  the  presentation 
(see  Figures  4  and  5) .  Helicopter  problem  results  (HSS  versus  HN 
versus  HSF) :  Enemy  Acts  factor  (F(2,162)  =2.1,  p  =  .13); 
Experienced  Man  factor  (F(2,162)  =  0.03,  p  =  .97).  River  problem 
results  (RSS  versus  RN  versus  RSF) :  Enemy  Strength  factor 
(F(2,37)  =  1.4,  p  =  .26);  Artillery  factor  (F(2,37)  =  0.02,  p  = 
.98);  River  Fordability  factor  (F(2,37)  =  0.01,  p  =  .99).  For 
both  problems,  the  evidence  from  those  factors  that  were  directly 
manipulated,  either  Objectively  or  Subjectively,  suggests  that 
when  using  the  Yes/No  and  Rating  methods  to  assess  relative 
importance,  people  have  a  global  rather  than  a  local 
interpretation  of  relative  importance. 

Halo  effects.  People  actively  construct  their  image  of  the  world. 
When  they  do  so,  we  might  expect  that  information  about  one 
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factor  may  influence  the  interpretation  of  information  about 
other  factors.  This  could  produce  a  "halo  effect"  with  relative 
importance  judgments:  as  the  probability  of  success  increases, 
those  factors  that  are  consistent  with  success  might  come  to  be 
seen  as  more  important,  and  factors  that  are  more  consistent  with 
failure  as  less  important.  Oppositely,  as  failure  becomes  more 
likely,  the  factors  consistent  with  failure  might  be  seen  as  more 
important.  Such  a  halo  effect  can  only  occur  given  a  local 
interpretation  of  relative  importance;  under  a  global 
interpretation,  the  probabilities  in  the  particular  case  should 
not  affect  the  relative  importance  of  the  factors,  which  should 
be  universal.  A  halo  effect  on  a  local  measure  of  relative 
importance  might  more  legitimately  occur  if  the  expectation  of 
success  is  manipulated  objectively,  i.e.,  by  changing  the  facts 
of  the  situation;  if  it  were  to  occur  when  the  mood  of  the 
presentation  is  manipulated,  it  would  suggest  people  are  too 
easily  swayed  by  subjective  aspects  of  the  situation. 

To  look  for  such  a  halo  effect,  the  factors  were  analyzed  a 
priori  to  see  whether  they  contribute  to  success  or  to  failure, 
or  whether  the  relation  is  uncertain  (it  is  not  known  what  will 
happen)  or  ambivalent  (unclear  whether  it  helps  or  hinders  the 
attainment  of  the  goal) .  For  the  Helicopter  problem,  the  factors 
in  the  story  that  contribute  primarily  to  failure  are  that  only 
one  patrol  is  sent,  that  the  patrol  is  small,  and  that  there  will 
be  no  air  reconnaissance.  The  choice  of  experienced  leader 
contributes  to  success.  The  true  location  of  the  crash  and  the 
response  of  the  enemy  are  uncertain.  The  roles  of  the  fog  and  the 
chosen  route  are  ambivalent  (fog  hinders  visibility  yet  provides 
cover;  the  direct  route  gets  the  patrol  rapidly  to  the  probable 
crash  site,  yet  can  be  predicted  by  the  enemy) .  For  the  River 
problem,  the  lack  of  friendly  air  support  contributes  to  failure. 
The  available  friendly  artillery,  the  fordability  of  the  stream, 
and  the  lack  of  enemy  air  support  contribute  to  success.  The 
enemy  strength,  the  ability  of  vehicles  to  approach  the  enemy 
positions,  the  extent  of  enemy  defenses,  and  whether  the  enemy 
has  positions  on  the  plain  are  all  uncertain.  Because  subjects 
did  not  indicate  any  significant  differences  in  the  relative 
importance  of  the  factors  that  were  changed  in  either  the 
Objective  or  Subjective  manipulation,  those  factors  are  included 
in  this  general  analysis. 

For  the  general  analysis,  the  factors  are  grouped  into  three 
sets:  tending  to  success,  tending  to  failure,  or 

uncertain/ ambivalent .  The  relative  importance  indices  (the  within 
subject  means  of  the  normalized  Yes/No  and  Rating  measures)  for 
each  of  the  factors  in  a  set  are  averaged,  within  subject,  to 
produce  each  of  the  three  factor  set  scores.  The  analysis  will 
look  at  the  across-subject  means  of  each  of  these  sets  as  the 
expectation  changes.  If  there  is  a  halo  effect,  it  will  be 
manifest  in  the  sets  of  factors  having  different  slopes  as  a 
function  of  expectation. 
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Let  us  first  consider  the  halo  effects  due  to  manipulation  of 
Objective  factors  pertinent  to  the  expectation  of  mission  success 
(Figures  6  and  7) .  First,  note  that  the  factors  that  contribute 
to  success  were  generally  considered  more  important  than  those 
that  contribute  to  failure.’  In  addition,  a  similar  pattern  was 
seen  for  both  problems:  as  the  expectation  of  success  decreased, 
the  ratings  of  the  success  and  the  failure  factors  converged. 

That  is,  the  judged  importance  of  the  factors  that  contribute  to 
success  was  larger  in  the  conditions  where  success  was  expected 
than  where  failure  was  expected  (for  both  problems) ,  and  the 
judged  importance  of  the  factors  that  contribute  to  failure  was 
larger  in  the  conditions  where  failure  was  expected  (especially 
for  the  River  problem) . 


Normal ized  Ralative  Importance 
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Factors 

Rmbiguous 

Factors 

Failure 

Factors 


Relative  Expectation  o-f  Success 
Established  in  the  Scenario 


Figure  6.  Relative  importance  of  general  factors,  Helicopter 
problem,  objective  conditions,  military  officers. 

This  convergence  embodies  the  halo  effect  of  interest,  and  is 
represented  by  the  interaction  term  in  a  repeated  measures 


9.  It  is  not  known  whether  this  is  due  to  the  way  the  problems  happen  to  have  been  written,  or  reflects 
people's  propensity  when  assessing  situations  to  attend  to  the  factors  that  lead  to  success  rather  than 
to  fai lure. 
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analysis  of  variance.  The  pattern  was  statistically  significant 
in  the  River  problem  (all  9  data  points:  F(4,210)  =  2.47,  p  = 
.046;  the  4  corner  points:  F(l,96)  =  4.21,  p  =  .043),  but  not  in 
the  Helicopter  problem  (all  9  data  points;  F(4,346)  =  1.30,  p  = 
.27;  the  4  corner  points:  F(l,64)  =  1.34,  p  =  .25).  As  argued 
above,  this  effect  indicates  that  the  relative  importance  • 
judgments  are  local  rather  than  global. 


No  rma 1 1 zed  Relative  I mpo  r  t  ance 


Relative  Expectation  of  Success 
Established  in  the  Scenario 


Figure  7.  Relative  importance  of  general  factors.  River  problem, 
objective  conditions,  military  officers. 

No  convergence  pattern  signifying  a  halo  effect  was  visible  in 
response  to  the  Subjective  manipulations,  and  the  interactions 
were  not  statistically  significant. 

In  summary,  subjects'  judgments  of  the  relative  importance  of 
factors  in  the  situation  did  not  change  significantly  when  those 
factors  were  explicitly  changed,  in  either  the  Objective  or 
Subjective  manipulation  conditions.  Thus,  direct  evidence  of  the 
local  use  of  the  relative  importance  judgments  was  not  seen. 
However,  a  hypothesized  halo  effect  was  found  in  response  to  the 
Objective  manipulations  of  expectation  of  success;  when  success 
was  more  likely,  factors  in  the  story  that  contribute  to  success 
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were  judged  more  important  and  factors  that  contribute  to  failure 
less  important.  This  pattern  was  observed  in  both  the  River 
problem  and  the  Helicopter  problem,  but  was  statistically 
significant  only  in  the  River  problem.  The  pattern  was  not  found 
with  the  Subjective  manipulations.  This  halo  effect  is  consistent 
with  a  local  rather  than  a  global  interpretation  of  relative 
importance . 

Probability  change  as  a  local  measure  of  relative  importance. 

Subjects'  judgments  using  the  Yes/No  and  Rating  methods  behaved 
consistent  with  a  global  interpretation,  except  for  the  halo 
effect  in  the  Objective  conditions  on  the  River  problem.  However, 
in  some  situations  it  may  be  an  advantage  for  a  measure  of 
relative  importance  to  be  local  rather  than  global.  The 
Probability  Change  measure  is  explicitly  local:  the  subject  is 
asked  to  say  how  much  a  hypothetical  change  from  the  level  of  a 
factor  specified  in  the  situation  would  change  the  probability  of 
success.  The  larger  the  change  in  probability,  the  more 
"important"  the  factor. 

Are  there  "local"  shifts  in  this  measure  of  relative  importance 
in  response  to  changes  in  the  situation,  as  would  be  expected?  We 
will  look  at  the  effects  of  the  Objective  and  Subjective 
manipulations  of  expectation  on  the  subjects*  Probability  Change 
judgments  for  the  individual  factors.’’  To  make  the  judgments 
easily  interpretable,  we  will  display  the  probability  change  (the 
probability  of  mission  success  if  the  factor  were  changed  as 
specified,  minus  the  original  estimate  of  the  probability)  rather 
than  the  normalized,  absolute  value  of  the  probability  change 
(which  serves  as  a  general  measure  of  relative  importance) . 

For  the  Helicopter  problem,  the  upper  section  in  Table  5  shows 
the  effect  of  Objectively  manipulating  the  expectation  of 
success.  None  of  the  8  factors  had  a  change  in  importance  that 
was  significant  at  less  than  p  =  .125.’^  For  example,  the 
importance  of  the  enemy's  actions  (the  probability  of  mission 
success  "if  the  enemy  were  able  to  see  where  the  helicopter  had 
crashed  and  had  already  sent  two  patrols")  did  not  change 
significantly  when  expectation  changed  from  success  to  failure 
(F(2, 127)  =  0.8,  p  =  .45) . 

When  the  expectation  of  success  on  the  Helicopter  problem  was 
Subjectively  manipulated  (lower  section  of  Table  5) ,  there  were 
several  changes.  For  example,  when  success  became  less  likely, 
the  anticipated  decrease  in  probability  of  mission  success  due  to 


10.  The  amount  of  probability  change  also  depends  on  the  size  of  the  change  in  the  factor.  Negating  or 
reversing  each  factor  is  one  way  to  establish  equal  amounts  of  change  (e.g.,  to  remove  artillery  support 
or  add  air  support),  so  that  differences  in  amount  of  probability  change  can  be  attributed  solely  to  the 
factors'  importance;  but  it  is  not  possible  to  "negate"  a  continuous  factor  such  as  enemy  strength. 

11.  Because  the  direction  and  extent  of  the  local  changes  depends  on  the  particular  hypothetical  level  that 
the  subject  was  asked  to  Judge,  it  does  not  make  sense  to  combine  the  individual  factors  into  sets. 

12.  Given  8  factors,  it  is  expected  that  one  would  be  significant  at  less  than  .125  by  chance. 
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the  hypothetical  choice  of  a  less-experienced  patrol  leader 
lessened  (from  -.104  to  -.026;  F(2,117)  =  3.7,  p  =  .028;  linear 
trend  F(l,118)  =  3.0,  p  =  .088).  Six  of  the  8  factors  were 
affected  at  p  <  .125,  and  5  had  linear  trends  that  were 
significant  at  p  <  .125. 

Table  5 

Effects  of  Objective  and  Subjective  manipulations  on  relative  importance 


of  factors,  measured  with  the  Probability  Change 
problem. 

Objective. 

Success  Neutral  Failure  General 

Factor  HOS  HN  HOF  N  F(2, 

method, 

Test 

N-2)  p 

,  Helicopter 

Test  of 
Linear  Trend 
F(1,N-1)  p 

Fog 

-.001 

.036 

.058 

128 

0.6 

.53 

1.2 

.27 

Enemy  Acts® 

-.322 

-.292 

-.242 

127 

0.8 

.45 

1.6 

.22 

Location 

-.174 

-.184 

-.156 

127 

0.2 

.82 

0.1 

.72 

Experncd  Man 

-.082 

-.129 

-.078 

125 

1.4 

.26 

0.0 

.  85 

Route 

-.099 

-.125 

-.065 

127 

1.1 

.32 

0.6 

.45 

One  patrol 

.  125 

.  116 

.  120 

127 

0.0 

.97 

0.0 

.93 

Small  patrol 

.030 

.055 

.064 

125 

0.2 

.78 

0.4 

.  51 

Aerial  recon 

.004 

.008 

.030 

122 

0.1 

.88 

0.2 

.  66 

Subjective. 

Factor 

Success 

HSS 

Neutral 

HN 

Failure 

HSF 

General 
N  F(2 

Test 
,N-2)  p 

Test  of 
Linear  Trend 
F(1,N-1)  p 

Fog 

-.011 

.036 

.082 

122 

]  .  5 

.24 

2.9 

.091 

Enemy  Acts* 

-.396 

-.292 

-.281 

121 

2.2 

.  114 

3 . 1 

.  079 

Location 

-.198 

-.184 

-.094 

121 

2.4 

.095 

3.7 

.  057 

Experncd  Man* 

-.104 

-.129 

-.026 

1 

3.7 

.028 

3.0 

.088 

Route 

-.117 

-.125 

-.033 

12! 

2.4 

.093 

2.7 

.  102 

One  patrol 

.157 

.  116 

.195 

120 

2.9 

.057 

1.0 

.  32 

Small  patrol 

.  126 

.055 

.  140 

119 

2.9 

.061 

0.1 

.73 

Aerial  recon 

.039 

.008 

.  108 

116 

2  _ 

.  130 

1.4 

.25 

“These  factors  were  directly  manipulated. 

For  the  River  problem  (Table  6) ,  the  Probability  Change  measure 
of  the  importance  of  enemy  strength  was  flawed  for  the  Objective 
variations,  as  discussed  above,  and  so  is  not  used  here.  The 
probability  change  judgments  of  4  of  the  other  7  factors  (upper 
section  of  Table  6)  were  affected  at  a  significance  level  of  p  < 
.143  (three  of  them  having  linear  trends  significant  at  p  < 

.143).  When  the  expectation  of  success  was  changed  through 
Subjective  manipulations,  on  the  other  hand,  only  the  artillery 
factor  showed  any  effect  (lower  section  of  Table  6) :  as  the 
expectation  of  success  decreased,  the  anticipated  decrease  in  the 
probability  of  mission  success  "if  the  brigade  were  not  able  to 
provide  the  battalion  with  field  artillery  support  during  the 
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attack”  lessened,  with  the  linear  trend  nearly  significant  at 
.057. 


Table  6 

Effects  of  Objective  and  Subjective  manipulations  on  relative  importance 
of  factors,  measured  with  the  Probability  Change  method.  River  problem. 

Objective.  Test  of 


Factor 

Success 

ROS 

Neutral 

RN 

Failure 

ROF 

General 
N  F(2, 

Test 

N-2)  p 

Linear 

F(1,N- 

Trend 
1)  P 

Enemy  Streng® 

•• 

- 

- 

- 

' 

- 

- 

Artillery 

-.243 

-.292 

-.195 

77 

1.3 

.28 

0.2 

.  63 

Fordability 

-.344 

-.383 

-  .215 

73 

2.8 

.069 

2.6 

.  113 

Veh  Passage 

-.272 

-.383 

-.220 

77 

3.8 

.027 

0.0 

.97 

Defen  Posit 

-.264 

-.263 

-.205 

78 

0.7 

.50 

1.1 

.30 

En  on  Ground 

-.195 

-.252 

-.130 

77 

1.9 

.  15 

0.5 

.48 

Friendly  Air 

.  113 

.067 

.269 

78 

10.9 

.000 

10.1 

.  0022 

Enemy  Air 

-.312 

-.425 

-.115 

74 

6.5 

.0025 

3.4 

.0695 

Subjective. 

Factor 

Success 

RSS 

Neutral 

RN 

Failure 

RSF 

General 
N  F(2, 

Test 
N-2)  p 

Test  of 
Linear  Trend 
FC-.N-l)  p 

Enemy  Streng® 

-.258 

-.188 

-.185 

37 

* 

o 

.40 

1.1 

.23 

Artillery® 

-.323 

-.292 

-.223 

37 

2.0 

.15 

3 . 9 

.  057 

Fordability® 

-.320 

-.383 

-.381 

37 

0.3 

.72 

0.5 

.  50 

Veh  Passage 

-.249 

-.383 

-.308 

37 

1.5 

.23 

0.6 

.44 

Defen  Posit 

-.296 

-.263 

-.258 

37 

0.2 

.84 

0.3 

.  59 

En  on  Ground 

-.246 

-.251 

-.169 

37 

1.6 

.22 

2 . 2 

.  14 

Friendly  Air 

.  106 

.  067 

.  127 

37 

0.9 

.41 

0.2 

.  64 

Enemy  Air 

-.356 

-.425 

-.294 

37 

2.1 

.  14 

1.0 

.33 

®These  factors  were  directly  manipulated. 

The  contrast  between  the  effects  of  the  Objective  and  Subjective 
manipulations  for  the  River  problem  is  most  evident  with  the 
Friendly  Air  Support  and  Enemy  Air  Support  factors  (Figure  8) .  In 
the  original  scenario,  it  was  anticipated  that  neither  side  would 
have  air  support.  When  the  reader  expected  failure  because  of  the 
objective  fact  that  the  enemy  was  at  90%  strength,  the  change  in 
mission  success  probability  if  "it  were  possible  for  LTC  Rubel  to 
be  supported  by  five  attack  helicopters"  increased  from  a  .11 
gain  (for  50%  strength)  or  a  .06  gain  (for  70%)  to  a  .27  gain 
(for  90%).  Similarly,  the  decrease  in  mission  success  probability 
if  "the  enemy  were  to  have  fighter  and  attack  helicopter  support" 
lessened  from  a  -.31  loss  (for  50%)  or  -.43  loss  (for  70%)  to  a 
-.12  loss.  When  the  expectation  of  failure  was  created  using  the 
subjective  manipulation  of  the  mood  of  the  scenario  presentation, 
the  changes  were  smaller  and  statistically  insignificant. 
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Probability  Change 
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- S'jbtect  i\e 

F  rndly  R: r 
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Enemy  Ri  ■' 

- Subiect ive 

Enemy  R: 


Expectation  of  Success 


Figure  8.  Effects  of  scenario  variations  on  raw  Probability 
Change  judgments. 

Unlike  a  global  measure  of  relative  importance,  which  should  not 
be  sensitive  to  changes  in  the  particular  levels  of  the  factors 
in  the  situation,  factor  importance  measured  by  a  local  relative 
importance  method  can  be  expected  to  shift  as  the  situation 
changes.  Shifts  in  response  to  changes  in  the  Subjective 
presentation  of  the  situation  would  not  be  desirable.  Change  in 
the  Objective  enemy  threat  in  the  River  problem  produced 
interpretable  changes  in  the  importance  of  non-manipulated 
factors,  as  measured  with  the  Probability  Change  method,  but 
changes  in  the  Subjective  presentation  of  the  situation  produced 
none.  In  contrast,  on  the  Helicopter  problem  the  Subjective 
changes  produced  nearly  significant  changes  while  the  Objective 
change  in  the  enemy's  behavior  produced  none.  These  unusual 
results  on  the  Helicopter  problem  are  consistent  with  the  finding 
(above)  that  the  Objective  manipulation  did  not  affect  its 
probability  of  success  while  the  Subjective  manipulation  did,  and 
with  findings  concerning  hindsight  bias  with  these  two  problems 
(Hamm,  1990d) . 
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Effects  of  expertise.  The  military  officer  subjects,  regardless 
of  branch,  are  familiar  with  the  general  battlefield  situation. 
They  have  all  had  formal  education  in  the  CGSC  courses;  in 
addition,  a  number  of  them  have  had  extensive  experience  as 
commanders  of  low  level  units  or  on  staffs  at  various  echelons, 
including  Vietnam  combat  experience  for  some  members  of  the 
reserves  and  national  guard  (who  are  older  than  the  active  army 
subjects) .  Therefore,  while  they  may  not  be  experts,  they  are 
competent  and  familiar  with  the  general  situation.  In  contrast, 
the  college  students  know  nothing  about  handling  battle 
situations,  other  than  what  is  available  in  the  general  culture. 
The  ROTC  students  have  more  exposure  to  and  interest  in  battle 
situations,  but  not  to  the  extent  of  the  career  military 
officers.  Therefore  differences  between  the  judgments  of  military 
officers  and  college  students  can  be  expected. 

Comparison  of  the  subjects'  probabilities  of  mission  success  for 
the  Helicopter  problem  (Figure  1)  shows  that  the  students 
generally  estimated  higher  probabilities  than  the  military 
officers  (F(  1,444)  =  14.6,  p  =  .000),  but  otheirwise  the  pattern 
of  probabilities  over  the  5  conditions  was  identical,  including 
the  nonmonotonic  response  to  the  objective  change  in  the  enemy 
threat.  In  fact,  the  ROTC  students  were  the  only  group  in  the 
study  for  whom  there  was,  on  average,  a  higher  probability  of 
success  when  the  enemy  presented  a  low  objective  threat  (.69,  n  = 
4)  compared  to  an  average  objective  threat  (.59,  n  -  6). 

To  determine  whether  there  were  any  differences  between  college 
and  CGSC  students  in  the  pattern  of  their  relative  importance 
ratings,  in  response  to  changes  in  expectation,  a  repeated 
measures  analysis  of  variance  (8  factors  by  three  conditions  by 
college/ CGSC)  was  done.  The  Factor  by  Condition  by  Group 
interaction  was  not  significant  for  either  the  objective  or  the 
subjective  manipulation  of  expectations.  Therefore  there  is  no 
evidence  that  people  with  military  experience  assess  the  relative 
importance  of  causal  factors  in  a  battlefield  situation  any 
differently  than  people  with  no  military  experience. 

Discussion 

Probability  of  success  insensitive  to  enemy  strength.  Subjects' 
judgments  of  the  probability  of  mission  success  did  not  shift  in 
the  predicted  manner  as  the  objective  enemy  threat  in  the 
situation  was  changed.  In  each  problem,  the  commander's  mission 
was  judged  to  have  a  higher  probability  of  success  in  the  medium 
threat  (Neutral)  condition  than  in  the  low  threat  (Objective 
Success)  condition.  Although  this  non-monotonic  relation  between 
enemy  threat  and  probability  of  success  fell  short  of 
conventional  statistical  significance  in  each  problem  (.10  <  p  < 
.15),  the  fact  that  the  same  pattern  occurred  on  both  the  River 
and  the  Helicopter  problems  (and  also  when  the  college  students 
did  the  Helicopter  problem)  means  it  should  be  taken  seriously. 

In  the  River  problem,  there  were  few  subjects  in  the  discrepant 
Neutral  condition,  so  the  non-monotonic  relationship  might  be 
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dismissed  as  a  chance  outcome.  However,  if  we  ignore  the  Neutral 
condition,  there  still  was  no  difference  between  the 
probabilities  of  success  when  the  enemy  in  the  River  problem  was 
at  50%  and  90%  strength,  nor  when  the  enemy  in  the  Helicopter 
problem  was  expecred  to  be  present  or  absent,  and  so  the 
conclusion  that  the  subjects'  probabilities  are  not  responding  to 
variation  in  an  important  aspect  of  the  situation  still  stands. 

The  finding  is  not  due  to  subjects  inferring  that  the  enemy  gains 
experience  as  it  loses  men.  A  plausible  line  of  reasoning  for  the 
River  problem  might  go:  If  the  war  has  just  started,  a  unit  at 
90%  strength  may  still  have  limited  experience;  by  the  time  it  is 
at  70%  strength  it  may  have  gained  experience  and  still  be  able 
to  fight;  and  when  at  50%  strength  it  is  no  longer  able  to  fight 
despite  its  experience.  The  combination  of  declining  strength  and 
increasing  experience  might  produce  an  enemy  which  offers  the 
greatest  threat  when  at  medium  rather  than  high  strength.  The 
implications  of  this  line  of  reasoning  for  our  side's  probability 
of  success,  however,  would  be  a  U-shaped  curve,  the  inverse 
pattern  from  what  was  observed  here. 

The  finding  that  military  officers'  numerical  probability 
judgments  are  not  accurate  may  have  limited  generality.  It  has 
been  demonstrated  here  with  between-subject  judgments,  where  one 
group  of  subjects  looks  at  Situation  A  and  another  looks  at 
Situation  B.  Their  judgments  may  be  more  accurate  when  each 
individual  compares  more  than  one  situation  (as  they  did  in  the 
Probability  Change  method;  see  below) .  Further,  the  Helicopter 
problem  is  unusual  in  that  it  involves  great  uncertainty.  One 
might  reasonably  argue  that  knowledge  about  the  enemy's 
competence  and  activity  level  is  not  pertinent  to  an  estimate  of 
the  probability  that  the  helicopter  would  be  rescued  in  this 
situation. 

But  it  is  impossible  to  ignore  the  insensitivity  to  the  enemy's 
strength  in  the  River  problem.  Common  sense  identifies  the  enemy 
as  a  very  important  element  of  the  situation.  In  addition,  the 
CGSC  students  have  been  taught  a  rule  of  thumb  which  explicitly 
relates  enemy  strength  to  the  probability  of  attack  success.  This 
is  the  force  ratio  rule:  the  ratio  of  attackers  to  defenders  must 
be  3  to  1  to  have  a  50%  chance  of  attack  success.  It  can  be  used 
for  determining  whether  one  should  seek  more  men  for  an  attack, 
or  as  a  tool  for  a  more  precise  prediction.  The  detailed 
information  required  for  the  application  of  particular  formulas 
expressing  the  force  ratio  rule  is  not  present  in  the  scenario, 
and  indeed  the  officers  may  be  aware  that  the  use  of  particular 
formulas  has  been  criticized  (Helmbold,  1969) .  Nonetheless,  they 
should  have  learned  to  think  intuitively  about  relative  strengths 
in  this  sort  of  situation,  so  that  differences  between  90%  and 
50%  enemy  strength,  all  other  factors  staying  constant,  would 
affect  their  judgments  of  the  probability  of  mission  success.  The 
question  is  whether  the  demonstrated  insensitivity  to  enemy 
strength  indicates  a  superficial  difficulty  in  using 
probabilities,  say  in  calibrating  them  when  making  a  one-shot 
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judgment,  or  a  deeper  difficulty  in  grasping  the  dynamics  of  a 
battlefield  situation. 

The  subjects'  performance  cn  the  relative  importance  judgment 
task  offers  evidence  pertinent  to  this  question.  Subjects 
revealed  a  "halo  effect"  when  the  enemy  strength  information  was 
varied  in  the  River  problem:  when  the  enemy  was  stronger,  they 
judged  the  failure-related  factors  to  be  more  important  and  the 
success-related  factors  to  be  less  important.  This  subtle  effect 
shows  that  the  military  officers  were  paying  fundamental 
attention  to  the  enemy's  strength.  Perhaps  this  is  at  the 
encoding  or  pattern  recognition  stage  rather  than  at  the  time  of 
judgment.  This  implies  that  the  officers'  inappropriate 
probability  judgments  are  due  to  their  difficulty  using 
probabilities  rather  than  to  an  inability  to  comprehend  the 
implication  of  enemy  strength  on  the  battlefield. 

Probability  of  success  sensitive  to  mood  of  situation 
presentation.  The  Subjective  conditions  varied  the  expectation  of 
success  through  changes  in  the  mood  of  the  scenario:  how  the 
environment  and  the  actors  were  described,  and  how  the  actors 
felt.  The  subjects'  probability  judgments  should  not  respond  to 
these  variations  in  mood,  and  indeed  they  did  not  in  the  River 
problem.  However,  the  manipulation  of  the  subjective  expectation 
of  success  through  mood  elements  in  the  presentation  of  the 
Helicopter  problem  significantly  affected  the  subjects'  judgments 
of  the  probability  of  mission  success. 

The  amount  of  information  manipulated  does  not  account  for  the 
effect  of  the  Subjective  manipulations.  In  the  Objective 
conditions  for  the  Helicopter  problem,  the  content  of  only  one 
paragraph  was  varied  (Table  2) ,  while  in  the  Subjective 
conditions,  six  paragraphs  were  varied  (Table  3).  Thus  the 
psychological  stimulus  was  more  powerful  in  the  Subjective 
conditions,  and  so  it  is  not  surprising  that  the  subjects 
responded  more  than  to  the  Objective  variation.  Applying  the  same 
argument  to  the  River  problem,  it  should  not  be  surprising  that 
the  Objective  variants  had  no  impact,  since  only  one  fact  was 
varied  (Table  2) .  But  by  the  same  token,  nine  pieces  of 
information  were  manipulated  in  the  Subjective  conditions  of  the 
River  problem  (Table  4) ,  yet  these  did  not  have  a  significant 
effect.  Therefore  the  amount  of  information  varied  between  the 
conditions  does  not,  by  itself,  explain  why  some  conditions  had 
effects  and  other  conditions  did  not. 

The  effect  of  the  subjective  variations  may  be  due  to  the  nature 
of  the  Helicopter  problem.  The  causal  factors  controlling  success 
here  may  be  so  ambiguous  that  the  subjects  feel  at  a  loss  to  make 
a  probability  estimate.  Indeed,  their  probability  judgments  were 
not  significantly  affected  by  the  changes  in  the  objective 
factor,  the  enemy's  activity  level.  In  ambiguous  conditions 
subjects  may  be  influenced  by  mood  information  that  they  would 
not  attend  to  otherwise. 
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Although  no  objective  information  varied  between  the  Subjective 
(presentation  mood)  conditions,  it  may  be  appropriate  for 
military  officers  to  attend  to  the  information  that  was  varied. 
Attending  to  the  way  other  people  say  and  do  things  may  be 
justified,  because  such  information  may  be  pertinent  on  a 
battlefield.  Exhausted  leaders  or  demoralized  subordinates  may 
not  fight  as  well.  In  most  real  world  situations,  there  is  a 
strong  association  between  what  people  are  excited  about  and  what 
is  important.  Further,  information  about  what  other  people  think 
and  feel  is  used  to  communicate  informally  about  what  groups 
should  be  attending  to  (Hamm,  1989) .  If  officers  had  to  explain 
everything  in  explicit  formal  statements,  it  would  take  them 
longer  (Linville,  Liebhaber,  Obermayer,  and  Fallesen,  1989)  and 
they  would  communicate  less.  Therefore  people's  use  of  subjective 
cues  is  not  in  itself  a  matter  of  concern,  because  such 
information  would  be  present  in  a  battlefield  and  would  probably 
provide  valid  cues  to  what  is  important  there.  It  would  be  a 
problem  only  if  these  subjective  cues  were  misleading. 

Finally,  it  might  be  argued  that  this  study's  demonstration  that 
people's  probability  estimates  respond  more  to  variations  in 
subjective  mood  than  to  changes  in  objective  facts  is  irrelevant 
to  our  understanding  of  military  officers'  thinking.  The  study 
involved  getting  people  to  read  stories.  It  did  not  put  them  on 
an  actual  or  a  simulated  battlefield.  Nor  did  it  give  them  the 
kind  of  written  communication  that  they  would  have  on  a 
battlefield,  i.e. ,  formally  structured  orders.  When  reading 
stories,  it  is  appropriate  for  people  to  pay  attention  to  the 
subjective  aspects  of  the  written  presentation.  In  fact,  literary 
conventions  train  us  to  attend  to  mood  information  in  a  story. 
Therefore  it  would  be  appropriate  for  subjects  to  be  affected  by 
the  mood  of  presentation  in  this  study,  but  this  says  nothing 
about  how  they  would  use  this  information  in  battle. 

This  argument  has  two  flaws.  First,  most  communication  in  a 
battle  does  not  involve  formal  written  orders  exclusively,  and 
therefore  study  of  the  effects  of  other  types  of  communication  is 
pertinent.  Second,  each  of  the  situations  was  reviewed  for 
realism  by  more  than  five  CGSC  instructors,  and  changes  they 
suggested  were  incorporated.  Therefore  the  scenarios  invoke  more 
than  just  the  subjects'  "story  reading"  ability;  they  also  invoke 
the  subjects'  knowledge  of  and  responses  to  battlefield 
situations. 

Different  probabilities  for  hypothetical  versus  given  situations. 

Subjects  estimated  different  probabilities  of  success  for  the 
same  situation,  depending  on  whether  it  was  the  given  situation 
that  they  were  to  assume  was  true,  or  a  hypothetical  situation 
different  in  one  feature  from  the  given  situation.  This  suggests 
people  use  a  cognitive  process  of  anchoring  on  their  probability 
assessment  of  the  given  situation  and  then  insufficiently 
adjusting  it  to  accomodate  the  change  in  the  situation.  By  this 
process,  a  consistency  principal  is  violated:  probability 
judgments  of  described  situations  should  be  the  same,  whether  the 
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description  be  "given”  or  "hypothetical".  This  inconsistency 
makes  it  difficult  for  numerical  probability  estimates  to  be 
useful  as  a  basis  for  action.  The  following  example  shows  why 
this  might  matter. 

Often  military  planning  is  done  assuming  a  particular  enemy 
course  of  action  will  be  done,  based  on  the  judgment  that  it  is 
their  most  likely  move  (Campbell  and  McKinney,  1990) .  A  number  of 
friendly  courses  of  action  will  then  be  developed,  and  a 
probability  of  success  assigned  to  each.  Then  the  planners  may 
consider  the  hypothesis  that  the  enemy  would  take  an  alternative 
course  of  action.  If  the  anchoring  and  adjustment  heuristic  that 
was  seen  in  this  study  is  used  to  assess  our  options  under  this 
revised  assumption,  it  is  possible  that  the  estimate  of  our 
probability  of  success  if  the  enemy  did  the  alternative  action 
would  be  too  close  to  the  estimate  of  our  probability  of  success 
if  they  did  the  assumed  action.  If  so,  the  ranking  of  the 
possible  friendly  courses  of  action,  in  order  of  their 
probabilities  of  success,  might  not  be  sufficiently  affected  by  a 
change  in  our  assumption  about  what  the  enemy  is  going  to  do. 

Thus  our  choice  of  course  of  action  might  be  based  on  wrong 
probability  assessments. 

College  students  more  optimistic  than  military  officers.  The  main 
difference  in  probability  assessments  between  the  military 
officers  and  the  college  students  was  that  on  the  average  the 
college  students  estimated  that  the  probability  of  success  was 
higher,  in  all  five  conditions  of  the  Helicopter  problem  (Figure 
1) .  This  may  reflect  the  effort  in  military  training  to  make 
officers  aware  of  what  could  go  wrong.  The  officers' 
probabilities  are  probably  better  calibrated,  but  we  can  not 
prove  this  since  there  is  no  "true  answer"  for  the  Helicopter 
problem.  No  differences  were  found  in  their  relative  importance 
judgments. 

Should  probabilities  be  used?  The  results  show  several  problems 
with  officers'  use  of  numerical  probabilities.  The  probabilities 
that  different  people  assign  to  the  same  situation  vary  greatly. 
Subjective  changes  in  the  presentation  of  the  situation  affect 
the  probabilities  although  they  should  not  (on  the  Helicopter 
problem) ,  and  objective  changes  affect  them  in  the  wrong  way 
(both  problems) .  People  seem  to  use  different  processes  when 
assigning  a  probability  to  the  given  situation  than  to  a 
hypothetical  situation  that  differs  in  one  feature.  Should  we 
respond  to  these  problems  by  giving  up  on  the  use  of  subjective 
judgments  of  probability  in  military  command  and  control,  and 
finding  alternative  ways  of  dealing  with  uncertainty?  Or  should 
we  try  to  help  military  officers  use  probabilities  better? 

The  argument  for  abandoning  subjective  probabilities  is  based  on 
widespread  demonstrations  that  people  make  errors  in 
probabilistic  and  statistical  reasoning  (see  Kahneman,  Slovic, 
and  Tversky,  1982) ,  and  that  experts  can  make  satisfactory 


decisions  without  using  probabilities  (e.g.,  Klein,  1989; 

Kuipers,  Moskowitz,  and  Kasserer,  1988). 

On  the  other  hand,  it  is  possible  to  improve  people's  use  of 
probabilities.  Several  alternatives  are  available.  Military 
officers  could  be  given  better  training  in  the  meaning  and  use  of 
probabilities.  The  successful  performance  of  medical  doctors 
(McClish  and  Powell,  1989)  and  weather  forecasters  (Murphy  and 
Winkler,  1977)  shows  that  accurate  and  well  calibrated 
probabilities  are  possible  in  some  domains.  Whether  they  are 
possible  in  military  contexts,  which  tend  to  present  unique 
rather  than  repetitive  situations,  remains  to  be  determined. 
Second,  the  use  of  verbal  rather  than  numerical  probabilities  has 
been  advocated  (Wallsten,  Budescu,  Rapoport,  Zwick,  and  Forsyth, 
1986) ,  because  their  inherent  vagueness  matches  the  ambiguity 
people  feel  when  judging  uncertainty,  but  helping  people 
communicate  reliably  with  these  would  require  training  or  aids 
similar  to  what  the  numerical  probabilities  require.  A  third 
alternative  is  to  provide  a  "warm-up"  before  requiring 
probability  judgments.  For  example,  a  fifteen  minute  reminder 
about  the  meaning  and  use  of  probabilities  can  lessen  some  of  the 
common  errors  in  statistical  reasoning  (Gebotys  and  Claxton- 
Oldfield,  1989) .  Finally,  provision  of  good  anchors,  for  example 
in  the  form  of  a  rule  of  thumb  for  calculating  the  probability  of 
success  which  can  then  be  adjusted  for  the  particular  case,  may 
allow  people  to  produce  more  accurate  probability  estimates  than 
if  they  simply  select  a  number  from  the  0  to  1  probability  scale. 
This  may  offer  improved  performance  even  though  we  have 
demonstrated  some  shortcomings  of  the  use  of  the  anchoring  and 
adjustment  strategy  for  making  probability  estimates. 

Evaluation  of  global  measures  of  relative  importance.  Military 
officers'  judgments  of  relative  importance  have  been  used  to 
produce  general  models  pertinent  to  long  term  planning  (Coates 
and  McCourt,  1976;  Dockery  and  Murray,  1987),  and  the  scope  of 
potential  applications  is  vast.  These  applications  depend  on  the 
relative  importance  judgments  being  general  or  global.  In  this 
study,  we  tested  whether  people  use  the  Yes/No  and  Rating  methods 
of  relative  importance  judgment  in  a  global  or  local  manner  when 
considering  a  particular  military  situation.  The  evidence  was 
mixed. 

The  most  obvious  opportunity  for  evidence  that  a  relative 
importance  measure  is  used  in  a  local  manner  was  in  the 
assessment  of  a  factor  that  varied  between  the  different 
occasions  in  which  importance  measurements  were  made.  In  the 
Objective  conditions,  subjects  rated  factors  (concerning  the 
enemy  threat)  that  were  varied  over  large  ranges.  This  provides  a 
strong  opportunity  for  a  local  interpretation  of  relative 
importance  to  manifest  itself.  But  no  significant  changes  in 
relative  importance  ratings  were  obseirved.  In  the  Subjective 
conditions,  subjects  rated  factors  that  were  varied  in  minor 
ways,  i.e.,  over  small  ranges  of  their  possible  values.  This 
provided  a  weak  opportunity  for  observation  of  local 
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interpretations.  Again,  there  were  no  significant  changes  in  the 
relative  importance  of  these  factors.  Thus,  there  was  no  sign 
that  people  use  these  relative  importance  measures  in  a  local 
manner,  in  the  most  obvious  places  for  such  a  sign  to  occur.  This 
suggests  that  people  are  interpreting  the  measures  in  a  global 
manner,  which  is  desirable  when  one  is  making  general  models. 

But  the  study  provided  another  opportunity  for  evidence  of  a 
local  interpretation  of  the  relative  importance  measures.  It  is 
possible  for  the  relative  importance  of  a  factor  to  change  in 
response  to  a  change  in  the  general  expectation  of  success,  when 
the  particular  factor  has  not  itself  been  changed.  Such  a  halo 
effect  was  observed  in  both  problems:  as  the  probability  of 
success  was  objectively  increased,  those  factors  that  are  seen  as 
contributing  to  success  were  judged  as  more  important,  and  those 
factors  seen  as  contributing  to  failure  were  judged  as  less 
important  (statistically  significant  for  the  River  problem  but 
not  for  the  Helicopter  problem) .  This  result  suggests  that 
subjects  are  using  the  relative  importance  measures  in  a  local 
manner  —  the  importance  depends  on  the  particular  situation 
(here,  its  probability  of  success)  rather  than  being  stable 
across  all  situations,  as  a  global  measure  of  importance  would 
be.  The  result  contradicts  the  conclusion  that  the  relative 
importance  measures  are  used  in  a  global  manner,  which  was 
reached  above  on  the  basis  of  the  lack  of  change  in  the 
importance  of  the  directly  manipulated  factors.  Explaining  this 
contradiction  is  a  matter  for  further  research,  and  it  suggests 
that  researchers  should  be  cautious  about  using  relative 
importance  judgments  to  make  general  models  unless  they  have 
taken  steps  to  assure  that  the  judgments  are  indeed  global. 

Local  measure  of  relative  importance.  A  local  measure  of  relative 
importance  could  be  useful.  For  example,  in  the  command  and 
control  of  military  forces,  one  might  need  to  be  able  to  say  very 
clearly  what  is  important  here,  and  it  would  not  matter  whether  a 
factor  would  be  important  in  other  situations.  In  this  view,  a 
global  expression  of  relative  importance  would  be  useful  in  a 
particular  situation  only  as  a  rule  of  thumb  if  one  must  act 
before  one  has  a  complete  understanding  of  the  situation.  The 
Probability  Change  measure  of  relative  importance  is  explicitly 
local  in  character.  Further,  if  we  do  not  take  the  absolute  value 
of  the  change  in  probability,  then  it  offers  information  about 
direction  as  well  as  extent.  In  this  way,  probability  change  is  a 
way  of  expressing  "impact”,  one  of  Goldstein's  (in  press) 
possible  forms  of  a  local  relative  importance  measure. 

To  determine  whether  the  Probability  Change  measure  of  relative 
importance  has  this  local  character,  we  investigated  whether  it 
changed  in  response  to  changes  in  the  situation.  We  expected  it 


13.  If  we  needed  to  make  probability  change  judgments  more  globally  useful,  we  could,  in  addition  to  taking 
the  absolute  value  of  the  change  in  probability  of  success  induced  by  a  change  in  the  situation, 
consider  changes  from  a  variety  of  starting  points  or  "original  situations",  and  establish  standard 
sizes  for  hypothetical  changes  by  reference  to  underlying  variability. 
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to  do  so  more  under  Objective  than  Subjective  changes  in  the 
situation.  There  were  opposite  results  for  the  River  and 
Helicopter  problems.  On  the  River  problem,  the  local  relative 
importance  measure  responded  to  Objective,  but  not  Subjective, 
changes.  On  the  Helicopter  problem,  it  was  sensitive  to  the 
Subjective  but  not  the  Objective  changes.  This  fits  with  other 
results  we  have  found  with  the  Helicopter  problem  (discussed 
above)  . 

The  Probability  Change  method  may  allow  for  more  accurate 
probability  judgments  because  it  uses  probability  change ,  rather 
than  stand-alone  judgments  of  probability.  While  people's 
assessments  of  the  probability  that  something  is  going  to  happen 
may  vary  widely,  they  may  be  more  accurate  in  their  assessment  of 
how  that  probability  will  change  if  an  aspect  of  the  situation 
changes.  This  may  be  true  even  though  judgments  of  probability 
change  are  not  perfect  because  of  the  involvement  of  the 
anchoring  and  adjustment  process,  as  demonstrated  above. 

The  reader  might  be  disturbed  by  the  apparent  contradiction 
between  the  conclusion  that  people's  probability  judgments  are 
flawed  (insensitive  to  important  variables;  sensitive  to 
unimportant  variables;  and  different  when  assessing  actual  and 
hypothetical  situations)  and  the  recommendation  that  the 
Probability  Change  method  (which  depends  on  probability 
judgments)  is  useful  as  a  local  relative  importance  measure.  One 
should  keep  in  mind  that  the  importance  judgments  probably  have 
similar  flaws  to  the  probability  judgments,  only  we  have  no  way 
of  discovering  these  flaws  because  there  are  no  standards  to 
which  relative  importance  judgments  can  be  compared.  The  laws  of 
probability  provide  a  way  of  providing  corrective  feedback  for 
training  probability  change  judgments,  and  there  is  no 
corresponding  corrective  for  the  general  judgments  of  relative 
importance . 

Success  factors  judged  more  important  than  failure  factors.  A 
final  concern  is  why  subjects  judged  the  failure  related  factors 
to  be  less  important  than  the  success  related  factors.  This  might 
be  due  to  arbitrary  choices  made  in  the  construction  of  the 
problems:  the  failure  related  factors  happened  to  be  ones  that 
people  think  are  less  important.  Or  it  may  reflect  people's 
tendency  to  think  in  terms  of  producing  success,  rather  than  in 
terms  of  avoiding  failure.  This  is  a  problem  for  future  research. 
It  would  be  easy  to  control  for  variation  in  the  size  of  the 
success  and  failure  factors  through  a  device  such  as  assigning 
different  subjects  to  different  sides  of  the  same  situation. 


38 


References. 


Anderson,  N.H.,  and  Zallnskl,  J.  (1988).  Functional  measurement 
approach  to  self-estimation  in  multiattribute  evaluation.  Journal 
of  Behavioral  Decision  Making,  1,  191-221. 

Campbell,  D.A. ,  and  McKinney,  R.W.  (1990).  Predictive 
intelligence:  An  old  lesson  unlearned.  Military  Review.  70(8, 

August) ,  50-58. 

Cheng,  P.W. ,  Holyoak,  K.J.,  Nisbett,  R.E.,  and  Oliver,  L.M. 

(1986) .  Pragmatic  versus  syntactic  approaches  to  training 
deductive  reasoning.  Cognitive  Psychology,  18,  293-328. 

Coates,  E.N.,  and  McCourt,  A.W.  (1976).  A  questionnaire-based 
analysis  of  order-of -battle  elements  (ARI  Technical  Paper  271) . 
Arlington,  VA:  Army  Research  Institute.  (AD  A021  956) 

Crumley,  L.M.  (1989) .  Review  of  research  and  methodologies 
relevant  to  army  command  and  control  performance  measurement  (ARI 
Technical  Report  825) .  Alexandria,  VA:  U.S.  Army  Research  Insti¬ 
tute  for  the  Behavioral  and  Social  Sciences.  (AD  A211  247) 

Dockery,  J.,  and  Murray,  E.  (1987).  A  fuzzy  approach  to 
aggregating  military  assessments.  International  Journal  of 
Approximate  Reasoning,  1,  251-271. 

Dreyfus,  H.L.,  and  Dreyfus,  S.E.  (with  Athanasiou,  T.)  (1986). 

Mind  over  Machine:  The  Power  of  Human  Intuition  and  Expertise  in 
the  Era  of  the  Computer.  New  York:  The  Free  Press. 

Fogarty,  W.M.  (1988) .  Formal  Investigation  into  the  C Ircumstances 
Surrounding  the  Downing  of  a  Commercial  Airliner  by  _he  USS 
Vincennes  on  3  July  1988.  Washington,  DC:  Department  of  Defense, 

July  28. 

Gebotys,  R.J.,  and  Claxton-Oldf ield,  S.P.  (1989).  Errors  in  the 
quantification  of  uncertainty:  A  product  of  heuristics  or  minimal 
probability  knowledge  base?  Applied  Cognitive  Psychology,  3, 

237-250. 

Goldstein,  W.M.  (In  preparation) .  Judgments  of  relative  importance  in 
decision  making:  Global  vs.  Local  interpretations  of  subjective 
weight.  Organizational  Behavior  and  Human  Decision  Processes . 

Goldstein,  W.M. ,  and  Beattie,  J.  (In  preparation) .  Judgments  of 
relative  importance  in  decision  making:  The  importance  of 
interpretation  and  the  interpretation  of  importance.  In  D.R. 

Brown  and  J.E.K.  Smith  (Eds.),  Frontiers  in  mathematical 
psychology.  New  York:  Springer-Verlag. 

Goldstein,  W.M.,  and  Mitzel,  H.C.  (In  preparation).  The  relative 
importance  of  relative  importance:  Inferring  other  people's 


39 


preferences  from  relative  importance  ratings  and  previous 
decisions.  Organizational  Behavior  and  Human  Decision  Processes. 

Hamm,  R.M.  (1989).  The  need  to  consider  modes  of  cognition  in 
designing  systems  that  require  distributed  decision  making.  In 
1989  IEEE  International  Conference  on  Systems ,  Man,  and 
Cybernetics,  Conference  Proceedings,  Volume  II,  pp  482-483. 

Hamm,  R.M.  (1990a).  Comparison  of  five  methods  for  expressing  the 
relative  importance  of  causal  factors  in  predicting  mission 
success.  In  Proceedings  of  Seventh  Annual  Workshop  on  Command  and 
Control  Decision  Aiding.  Dayton,  OH:  Air  Force  Institute  of 
Technology. 

Hamm,  R.M.  (1990b) .  Comparison  of  five  methods  for  expressing 
relative  importance  in  a  causal  prediction  context.  Fort 
Leavenworth,  KS:  U.S.  Army  Research  Institute. 

Hamm,  R.M.  (1990c).  Expressions  of  relative  importance.  In  I.B. 
Turksen  (Ed.),  Proceedings  of  NAFIPS * 90:  Quarter  century  of 
fuzziness  (Volume  II,  pp  382-385).  Toronto:  North  American  Fuzzy 
Information  Processing  Society. 

Hamm,  R.M.  (1990d) .  Hindsight  bias  in  relative  importance 
judgment.  Paper  presented  at  meetings  of  the  Psychonomic  Society, 
New  Orleans,  November. 

Hamm,  R.M.,  Bursztajn,  H.,  Mills,  M.,  Appelbaum,  P.,  and  Gutheil, 
T.G.  (1981) .  Regression  and  introspection  in  the  analysis  of 
clinical  decision  making:  The  case  of  the  3-day  paper.  Boston: 
Massachusetts  Mental  Health  Center. 

Hammond,  K.R.,  Hamm,  R.M.,  Grassia,  J.L.,  and  Pearson,  T.  (1987). 
Direct  comparison  of  the  efficacy  of  intuitive  and  analytical 
cognition  in  expert  judgment.  IEEE  Transactions  on  Systems,  Man, 
and  Cybernetics,  SMC- 17 ,  753-770. 

Helmbold,  R.L.  (1969) .  Probability  of  victory  in  land  combat  as 
related  to  force  ratio  (Report  P-4199).  Santa  Monica,  CA:  Rand 
Corporation. 

Hogarth,  R.M.  (1987).  Judgment  and  choice:  The  psychology  of 
decision  (2nd  Edition) .  New  York:  Wiley. 

Kahneman,  D. ,  Slovic,  P. ,  and  Tversky,  A.  (Eds.,  1982).  Judgment 
under  Uncertainty:  Heuristics  and  Biases.  New  York:  Cambridge 
University  Press. 

Klein,  G.A.  (1989).  Strategies  of  decision  making.  Military 
Review,  May,  56-64. 

Kuipers,  B.,  Moskowitz,  A.J.,  and  Kassirer,  J.P.  (1988).  Critical 
decisions  under  uncertainty:  Representation  and  structure. 
Cognitive  Science,  12 ,  177-210. 


40 


Lichx: 'Astein,  S.,  Fischhoff,  B.,  and  Phillips,  L.D.  (1982). 
Calibration  of  probabilities:  The  state  of  the  art  to  1980.  In  D. 
Kahneman,  P.  Slovic,  and  A.  Tversky  (Eds.),  Judgment  under 
Uncertainty:  Heuristics  and  Biases  (pp  306-335) .  New  York: 

Cambridge  University  Press. 

Linville,  J.M.,  Liebhaber,  M.J.,  Obermayer,  A.H.,  and  Fallesen, 

J.J.  (1989) .  Computer-mediated  group  processes  in  distributed 
command  and  control  systems:  Dyad  shared  work  (ARI  Technical  Report 
842) .  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences.  (AD  A211  337) 

McClish,  D.K.,  and  Powell,  S.H.  (1989).  How  well  can  physicians 
estimate  mortality  in  a  medical  intensive  care  unit?  J.  Medical 
Decision  Making ,  9,  125-132. 

Murphy,  A.H.,  and  Winkler,  R.L.  (1977).  Can  weather  forecasters 
formulate  reliable  probability  forecasts  of  precipitation  and 
temperature?  National  Weather  Digest.  2,  2-9. 

Nisbett,  R.E.  (1988).  Testimony  before  U.S.  House  of 
Representatives,  Committee  on  Armed  Services,  on  the  subject  of 
the  USS  Vincennes  Downing  of  Iran  Air  Flight  655.  Washington,  DC: 
American  Psychological  Association,  October  6. 

Olmstead,  J.A.,  Christensen,  H.E.,  and  Lackey,  L.L.  (1973). 

Components  of  organizational  competence :  Test  of  a  conceptual 
framework  (HumRRO  TR-73-19) .  Alexandria,  VA:  Human  Resources 
Research  Organization. 

Pennington,  N.,  and  Hastie,  R.  (1988).  Explanation-based  decision 
making:  Effects  of  memory  structure  on  judgment.  Journal  of 
Experimental  Psychology:  Learning.  Memory ,  and  Cognition.  14 , 

521-533. 

Poses,  R.M. ,  Bekes,  C.,  Copare,  F.J.,  and  Scott,  W.E.  (1990). 

What  difference  do  two  days  make?  The  inertia  of  physicians' 
sequential  prognostic  judgments  for  critically  ill  patients.  J. 
Medical  Decision  Making,  10.  6-14. 

Reilly,  B.A. ,  and  Doherty,  M.E.  (1989).  A  note  on  the  assessment 
of  self-insight  in  judgment  research.  Organizational  Behavior  and 
Human  Decision  Processes.  44 ,  123-131. 

Schmitt,  N.,  and  Levine,  R.L.  (1977).  Statistical  and  subjective 
weights:  Some  problems  and  proposals.  Organizational  Behavior  and 
Human  Performance,  20,  15-30. 

Stockfisch,  J.A.  (1975).  Models,  data,  and  war:  A  critigue  of  the 
study  of  conventional  forces  (Report  R-1526-PR) .  Santa  Monica, 

CA:  Rand  Corporation. 

Sulzen,  R.H.  (1987,  May).  Winning  with  tactical  engagement 
simulation.  Military  Review,  9-19. 


41 


Tape,  T.G.,  and  Wigton,  R.S.  (1989).  Medical  students’  and 
residents'  estimates  of  cardiac  risk.  J.  Medical  Decision  Making, 
9,  170-175. 

Tolcott,  M.A. ,  Marvin,  F.F.,  and  Lehner,  P.E.  (1989).  Expert 
decisionmaking  in  evolving  situations.  IEEE  Transactions  on 
Systems ,  Man,  and  Cybernetics,  19 ,  606-615. 

Trabasso,  T.,  and  Sperry,  L.L.  (1985).  Causal  relatedness  and 
importance  of  story  events.  Journal  of  Memory  and  Language,  24 , 
595-611. 

Wallsten,  T.S.,  Budescu,  D.V. ,  Rapoport,  A.,  Zwick,  R. ,  and 
Forsyth,  B.  (1986).  Measuring  the  vague  meanings  of  probability 
terms.  Journal  of  Experimental  Psychology;  General ,  115 ,  348-365. 


42 


Appendix  A. 

Text  of  Helicopter  Problem. 

Please  read  the  following  problem.  You  will  be  asked  to  evaluate  the 
commander's  plan  by  estimating  its  probability  of  success  and  suggesting 
improvements.  You  will  also  be  asked  to  rate  the  importance  of  different 
factors  facing  the  commander. 

Dov/ned  Helicopter  Problem 

Captain  Smith  is  a  rifle  company  commander  in  a  U.S.  campaign  to 
assist  a  third  world  government  fighting  against  well  armed  insurgent 
forces.  His  company  has  responsibility  for  an  area  of  operations  on  the 
western  edge  of  a  valley  that  runs  north  and  south  [see  map  on  next  page]. 
The  international  border  is  several  kilometers  farther  east.  The  enemy's 
bases  are  on  the  other  side  of  the  border,  and  for  political  reasons  no  major 
offensive  has  been  launched  against  them.  The  enemy  is  frequently  active 
on  the  eastern  edge  of  the  valley,  separated  from  Smith’s  company  by  a 
4000  meter  area  that  is  difficult  to  move  through  due  to  swamps,  streams 
and  ravines,  dense  jungle,  and  no  roads.  There  has  been  little  enemy 
activity  in  this  sector,  other  than  probes  and  occasional  contacts, 
over  the  Irst  several  days.  [*] 

Intelligence  does  not  think  that  an  attack  is  imminent,  but 
warns  that  the  company  should  be  alert  for  reconnaissance 
patrols  and  probes. 

It  is  a  half  hour  after  dawn.  Weather  has  been  heavily  overcast  for 
two  days,  and  this  morning’s  thick  fog  is  expected  to  stay  on  or  near  the 
ground,  with  intermittent  light  rain,  all  day.  A  few  minutes  ago  a 
helicopter  was  heard  in  the  valley,  approaching  from  the  south.  Although  it 
came  close,  it  was  not  visible  due  to  the  fog.  None  of  Capt.  Smith’s  people 
could  identify  its  type  from  the  sound.  Observeis  heard  the  helicopter 
crash  approximately  2000  meters  in  front  of  the  forward  observation  post, 
about  midway  through  the  valley. 

A  call  to  headquarters  produced  no  information  about  the  identity 
of  the  helicopter.  Smith  was  ordered  to  ’’Check  it  out.  It  might  be  one  of 
ours,  and  if  so  they  are  probably  going  to  need  some  help.  The  Assistant 
High  Minister  of  Defense  was  scheduled  to  inspect  the  area  north  of  us 
today.  Maybe  he  got  lost.  If  it  is  the  enemy’s,  we  might  get  important 
information  from  the  survivors  or  tne  wreckage."  [*] 

Capt.  Smith  tells  Lt.  Wilson,  one  of  his  platoon  leaders,  to  send  a 
squad  to  find  the  helicopter,  aid  the  injured,  protect  them  and  bring  them 
back  if  they  are  ours,  and  to  get  prisoners  and  information  if  they  are  the 
enemy.  Capt.  Smith  notices  that  Lt.  Wilson  is  eager  to  lead  the 
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patrol  himself,  but  Wilson  decides  to  send  a  ten-man  squad  under  Sgt. 
Jones,  who  has  much  more  experience  moving  about  in  the  area.  Jones 
has  led  20%  of  the  patrols  in  this  area  since  the  company  took  up  its 
position,  and  has  debriefed  most  of  the  other  patrols.  Because  of  the 
urgency  of  the  situation  (probable  injuries,  and  the  possibility  that  the 
enemy  will  get  there  first),  Capt.  Smith  instructs  Wilson  that  the  patrol 
should  take  the  most  direct  route  to  the  likely  crash  site.  [*] 
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Figure  A-1. 
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Appendix  B. 

Text  of  River  Problem. 

Please  read  the  following  problem.  You  will  be  asked  to  evaluate  the 
commander's  plan  by  estimating  its  probability  of  success  and 
suggesting  improvements.  You  will  also  be  asked  to  rate  the 
importance  of  different  factors  facing  the  commander. 

River  Crossing  Problem 

Lieutenant  Colonel  Rubel  commands  a  mechanized  infantry 
battalion  that  is  located  in  the  front  line.  The  battalion  moved  into  its 
current  position  unopposed  in  the  early  evening.  It  is  2300,  and  LTC 
Rubel  is  finalizing  his  plans  for  the  next  day's  battle,  [*]  The  enemy 
is  across  a  small  river  [see  map  on  next  page],  in  several  small 
wooded  hills  a  kilometer  back  from  the  river.  They  too  have  just 
arrived  in  their  positions  within  the  past  several  hours. 

LTC  Rubel's  orders  are  to  establish  a  bridgehead  on  the  other 
side  of  the  river  by  destroying  or  driving  off  the  enemy  forces  and 
taking  the  hill  marked  434,  and  clearing  the  objective  of  enemy.  The 
attack  is  to  start  by  0700,  and  Rubel  is  to  take  Objective  Tiger  (Hill 
434)  and  clear  the  area  by  1200,  and  prepare  a  hasty  defense.  The 
purpose  is  to  allow  a  larger  force  to  ford  the  river  here  and  attack  to 
the  northwest. 

Since  the  arrival  of  the  order,  LTC  Rubel's  staff  has  presented 
him  with  the  following  information. 

River  crossings.  Scouts  report  the  river  is  easily  fordable  at 
points  X,  Y,  and  Z,  with  low  firm  banks  and  a  firm  bottom. 

Weather.  No  rain  is  expected  for  24  hours.  Wind  will  be  light, 
from  the  southwest,  and  so  smoke  can  be  used. 

Terrain.  Hill  434  though  high  is  fairly  accessible.  There  are 
no  cliffs,  and  the  wooded  slopes  are  not  so  steep  that  people  will 
have  trouble  climbing  them,  although  due  to  the  trees  and  the  slope, 
it  is  not  anticipated  that  vehicles  can  be  used  in  the  assault  on  the 
hill.  The  area  between  the  river  and  the  bottom  of  Hill  434  is  lightly 
wooded  with  numerous  old  logging  trails.  Cross-country  mobility  is 
poor.  On  the  east  and  west  shoulders  of  Hill  434,  the  ground  rises 
gently  to  the  ridge  line.  There  are  somewhat  steeper  slopes  in  the 
middle  of  the  hill.  The  upper  part  of  the  hill  approaches  a  40  degree 
slope,  with  a  minimal  number  of  rocky  outcroppings. 
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Enemy  forces.  Intelligence  reports  that  the  enemy  is  a 
motorized  rifle  battalion  equipped  with  BTRs  (wheeled  armored 
personnel  carriers),  with  3  companies,  two  at  the  front  and  one  in 
reserve  [placed  approximately  as  shown  on  the  map],  and  they  have 
been  attrited  to  70%  strength.  After  retreating  into  the  area 
following  today's  battles  to  the  east,  the  enemy  has  started  preparing 
defensive  positions.  They  are  not  likely  to  attack  here  tonight 
or  tomorrow.  It  can  be  assumed  that  there  are  at  present  no  mines 
in  the  river  valley,  since  the  enemy  has  not  had  time  to  place  any, 
and  friendly  forces  have  not  left  any  there  earlier. 

The  three  hills  provide  the  enemy  with  good  observation,  and 
they  will  be  able  to  bring  fire  from  both  mortar  and  artillery  to 
bear  on  the  attack.  The  battalion  will  have  support  from  the 
regiment's  organic  artillery  battalion,  and  it  is  likely  they  will  have 
additional  artillery  support.  It  is  assumed  that  the  enemy  has  no 
helicopter  or  air  support  available  for  use  in  this  sector,  due  to  losses 
in  previous  weeks  and  because  their  air  resources  arc  being 
concentrated  for  a  major  battle  being  fought  to  the  north  and  east. 

Friendly  forces.  Rubel's  task  force  (three  rifle  companies 
and  one  cross  attached  tank  company)  are  at  85%  strength.  Each  of 
the  mechanized  rifle  companies  has  10  M2  Bradley  fighting  vehicles. 
The  tank  company  has  11  Ml  tanks.  Only  light  wind  is  expected,  and 
so  it  will  be  possible  to  generate  smoke  to  limit  visibility  during  the 
river  crossings.  Smoke  is  available  from  the  6  mortars  of  the 
headquarters  company,  from  the  two  batteries  of  155’s  that  have 
been  made  available  to  the  battalion  during  the  morning  hours,  and 
from  smoke  generators  that  have  been  provided.  Unfortunately,  no 
helicopter  or  air  support  is  available.  A  scout  platoon  is  forward  for 
screening  and  reconnaissance.  They  report  some  enemy  activity  in 
the  plain  between  Hill  434  and  the  river. 

Rubel's  plan  is  to  slow  the  enemy's  preparations  with 
harassing  and  interdicting  artillery  fire  and  occasional  mortar  fire 
throughout  the  night.  This  should  make  it  more  difficult  for 
them  to  place  obstacles  in  the  fords  or  mine  the  approaches  to  Hill 
434,  as  well  as  slowing  their  defensive  preparations.  The  tank 
company  (Blue  Company  C)  and  Company  B  will  take  positions  on 
Hills  340  and  312,  from  which  they  will  be  able  to  apply 
overwatching  direct  suppressive  fire  during  the  first  phase  of  the 
attack. 
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At  0500,  the  scouts  will  cross  the  river  (under  smoke)  and 
deploy  in  the  plain,  seek  to  discover  enemy  positions,  check  the  route 
that  the  armored  personnel  carriers  will  use  later,  and  take  up 
positions  to  protect  the  flanks  of  the  coming  attack.  Then  while 
Companies  B  and  C  supply  suppressive  fire  and  the  artillery  smokes 
the  enemy  on  Hill  434  to  prevent  their  observations.  Companies  A 
and  D  will  cross  ford  X  under  smoke  and  move  as  rapidly  as 
possible  to  attack  the  western  half  of  the  hill.  As  they  cross  Phase 
Line  CAT,  the  artillery  will  shift  target  to  the  top  and  the  eastern  half 
of  Hill  434,  to  prevent  the  forces  there  from  firing  on  the  two 
companies.  The  tank  company  (C)  will  continue  to  provide  direct 
suppressing  fire  to  the  western  half  of  the  hill,  while  Company  B  will 
prepare  to  ford  the  river. 

While  Companies  A  and  D  attack  the  western  half  of  the  Hill 
434,  Company  B  will  ford  the  river  and  move  into  position  to  either 
support  them  or  attack  the  eastern  half  of  the  hill.  The  artillery 
will  shift  to  suppressing  the  rear  enemy  forces  on  Hills  230  and 
318.  When  Companies  A,  B,  and  D  are  consolidating  on  the  objective, 
the  tank  company  will  move  down  from  Hill  312  and  prepare  to  ford 
and  protect  Hill  434  from  counterattack, 

[*] 
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Appendix  C. 

Probability  Change  questions  for  Helicopter  Problem. 

Probability  Change  Importance  Rating. 

The  success  of  Capt.  Smith's  plan  depends  on  several 
important  factors.  Please  consider  the  following  list,  and  estimate 
what  the  probability  of  mission  success  would  be  if  the  factor  were 
to  change  as  described.  Please  give  specific  probabilities.  Each  of  the 
changes  should  be  considered  independently,  leaving  all  other 
features  as  described  originally. 

Refer  to  the  "Plan  Evaluation"  question  above  to  review  your 
estimate  of  the  probability  that  the  patrol  will  succeed  in  finding  the 
helicopter  and  returning  with  the  injured  or  prisoners,  given  current 
conditions  and  Capt.  Smith's  original  plan. 

What  would  be  the  probability  of  mission  success  if  the  fog  were  to 
lift  so  that  both  Sgt.  Jones'  patrol,  and  the  enemy,  had  good  visibility? 

What  would  be  the  probability  of  mission  success  if  the  enemy 
were  able  to  see  where  the  helicopter  crashed  and  has  already  sent 
two  patrols?  _ 

What  would  be  the  probability  of  mission  success  if  the  actual  crash 
site  were  more  than  1000  meters  from  the  estimated  crash  site? 


What  would  be  the  probability  of  mission  success  if  Lt.  Wilson  had 
led  the  patrol  himself  instead  of  assigning  Sgt.  Jones  who  has  more 
experience  with  the  terrain? _ 

What  would  be  the  probability  of  mission  success  if  Captain  Smith 
had  ordered  the  patrol  to  take  a  safe  but  slow,  indirect  route  instead 
of  the  most  direct  route  to  the  estimated  crash  site?  _ 

What  would  be  the  probability  of  mission  success  if  Captain  Smith 
had  sent  two  separate  patrols  rather  than  just  one? _ 

What  would  be  the  probability  of  mission  success  if  Captain  Smith 
had  sent  a  large  patrol  (30  men  rather  than  11)? _ 

What  would  be  the  probability  of  mission  success  if  Captain  Smith 
had  decided  to  request  aerial  reconnaissance  before  sending  out  the 
patrol? 
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Appendix  D. 

Probability  Change  questions  for  River  Problem. 

Probability  Change  Importance  Rating. 

The  outcome  of  LTC  Rubel’s  attack  plan  depends  on  several 
important  factors.  Please  consider  the  following  list,  and  estimate 
what  the  probability  of  attack  success  would  be  if  the  factor  were  to 
change  as  described.  Please  give  specific  probabilities.  Each  of  the 
changes  should  be  considered  independently,  leaving  all  other 
features  as  described  originally. 

Refer  to  the  "Plan  Evaluation"  question  above  to  review  your 
estimate  of  the  probability  that  the  attack  will  succeed  in  taking  Hill 
434  by  1200  given  current  conditions  and  LTC  Rubel's  original  plan. 

What  would  be  the  probability  of  attack  success  if  the  enemy  in  fact 
were  at  70%  strength?  _ 

What  would  be  the  probability  of  attack  success  if  the  brigade  were 
not  able  to  provide  the  battalion  with  field  artillery  support  during 
the  attack?  _ 

What  would  be  the  probability  of  attack  success  if  the  river  could 
not  in  fact  be  forded  by  the  IvI2s?  _ 

What  would  be  the  probability  of  attack  success  if  the  roads  in  the 
area  between  the  river  and  Hill  434  were  mined  and  blocked? 


What  would  be  the  probability  of  attack  success  if  the  enemy 
turned  out  to  have  completed  preparing  its  defensive  positions  on 
Hill  434?  _ 

What  would  be  the  probability  of  attack  success  if  the  enemy 
turned  out  to  have  positions  in  the  area  between  the  river  and  Hill 
434? _ 

What  would  be  the  probability  of  attack  success  if  it  were  possible 
for  LTC  Rubel  to  be  supported  by  five  attack  helicopters? _ 

What  would  be  the  probability  of  attack  success  if  the  enemy  were 
to  have  fighter  and  attack  helicopter  support? _ 
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